ci: generate annotations for Lua types

[ofseed]

Automatically generate Lua types from the LSP server settings JSON schema

Modified from neoconf.nvim

[igorlfs]

Hey, thanks for looking into this.

With #4010 I had a similar goal, although with a much more limited scope.

When this PR is finished, the changes, from a user's perspective, are:

• Built-in configurations will now use custom types (e.g., instead of generic vim.lsp.Config, would use lspconfig.tsgo for tsgo, etc)
• These custom types will be exposed, so they can also be used elsewhere (i.e., user's config)

Are these assumptions correct?

[ofseed]

• Built-in configurations will now use custom types (e.g., instead of generic vim.lsp.Config, would use lspconfig.tsgo for tsgo, etc)

Since the previous way to enable lspconfig followed the lspconfig.tsgo.setup pattern, neoconf was able to configure the type for keys like tsgo. This allowed users to avoid manual annotations once the generated types were loaded. However, the current vim.lsp.config chooses to pass the server name as a positional argument rather than a key, so this method might not be effective—it might still work via ---@overload, but I haven't tried that yet.

I don't intend to deprecate @type vim.lsp.Config, but rather to provide compatibility. My current preliminary plan is to provide a series of separate @class lspconfig.tsgo.settings, so that manual annotation is at least possible.

• These custom types will be exposed, so they can also be used elsewhere (i.e., user's config)

Yes, this is a very important objective. In fact, neoconf itself doesn't need these type definitions; it generates them purely as a feature to make it convenient for users to use these types within their own configurations.

[ofseed]

I just noticed an issue: since the JSON schemas are all JSON with comments rather than standard JSON, vim.json cannot be used here. Are there any plans to have vim.json support jsonc? Otherwise, a jsonc parser must be added here. cc @justinmk

[justinmk]

Are there any plans to have vim.json support jsonc?

I'm favor of that but we don't have a tracking issue for it, nor in https://github.com/openresty/lua-cjson .

If it's not too horrible, this may even justify forking cjson if it won't be accepted upstream.

cc @skewb1k

[arkriny]

Are there any plans to have vim.json support jsonc?

I think it's reasonable to have a flag for that.

I'm favor of that but we don't have a tracking issue for it, nor in https://github.com/openresty/lua-cjson .

If it's not too horrible, this may even justify forking cjson if it won't be accepted upstream.

We can take a similar approach to neovim/neovim#35574 and neovim/neovim#35424 by opening PRs to both upstream and neovim, and being okay with merging into neovim first. If upstream is not interested, we can certainly diverge. Additionally, I recall some discussions on Matrix regarding hard-forking due to API differences and other factors, so I don't think that's a problem.

[arkriny]

I just noticed an issue: since the JSON schemas are all JSON with comments rather than standard JSON, vim.json cannot be used here.

While it's cool to support comments in vim.json and I'm working on the patch, I don't quite understand why its needed for JSON schemas? It seems that all schemas are valid JSONs, and the specification even introduces the $comment field, which is intended to be ignored semantically. Probably it wouldn't be necessary if the comment syntax was allowed.

local function readfile(path)
  local file = io.open(path, "r")
  if not file then
    return nil
  end
  local content = file:read("*a")
  file:close()
  return content
end

local dir = "./schemas"
local files = vim.fn.readdir(dir)
for _, file in ipairs(files) do
  local file_path = vim.fs.joinpath(dir, file)
  local content = readfile(file_path)
  vim.json.decode(content)
end

Running this in the root of https://github.com/folke/neoconf.nvim succeeds (introducing comment manually will cause it to fail).
I was thinking that in neoconf, jsonc is necessary to support comments in the user's config file. Am I missing something?

[ofseed]

You are correct. I noticed that neoconf uses a JSONC decoder for JSON schemas, which led to my misunderstanding that JSON schemas could be JSONC; they should strictly be standard JSON.

The reason neoconf uses JSONC parsing here should because this wrapped function is also used to parse settings.json, which is allowed to be JSONC. Therefore, for this PR, using vim.json is sufficient.

[ofseed]

I have completed the essential refactoring; this PR is now in a usable state.

[justinmk]

The lua/lspconfig/types/lsp.lua file is big and will keep growing. Does it avoid a perf issue if we split it into a bunch of files per-config?

[justinmk]

is @schema a LuaLS/emmylua feature? should we mention it in the readme?

[ofseed]

This is just something I defined myself so that we can annotate the JSON schema while adding a server. I may need to mention it in the contributing guide or somewhere similar.

However, emmylua-analyzer-rust does have a @schema （https://github.com/EmmyLuaLs/emmylua-analyzer-rust/pull/952）field that functions like @type, using a remote JSON schema as the @type. But it seems to support a limited version of JSON schema. I haven't tried it yet.

[ofseed]

The lua/lspconfig/types/lsp.lua file is big and will keep growing. Does it avoid a perf issue if we split it into a bunch of files per-config?

I haven't noticed any performance issues during my own use, but the problem is that the default preloadFileSize field in the Lua LS setting is 500 KB, while the lsp.lua type file generated by this script is around 750 KB, causing the file to be ignored by default. It might be worth splitting it up or updating the preloadFileSize setting in lsp/lua_ls.lua.

I was just about to apply @type lspconfig.settings to the settings under lsp/, but then I noticed that some of the generated results seem to be invalid, maybe because the provided link is invalid. I probably need to double-check them before requesting a review again.

[justinmk]

the codespell CI should ignore types/ since we don't control the spelling there.
the other failing lints are presumably legitimate

[justinmk]

this reminds me we need neovim/neovim#31930 to make writefile() less annoying :)

[justinmk]

just fyi, we have vim.fs.rm() now too

[ofseed]

I just noticed that I lost the Co-authored-by in the commit during the final rebase; thanks to @folke for providing the original version!