Goal of Code Search for GNOME Builder is to provide ability to search all symbols in project fuzzily and jump to definition from reference of a symbol in GNOME Builder. For implementing these we need to have a database of declarations. So I created a plugin called ‘Indexer’ in Builder which will extract information regarding declarations and store them in a database.
What information needs to extracted from source code and how?
Since we need to search symbols by their names, names of all declarations needs to be extracted. And we also need to jump to definition of a symbol from a reference of that one, so keys of all global declarations also needs to be extracted. Note that keys of local declarations needs not to be extracted because whenever we want to jump to local definition of a symbol from its reference, that definition will be in current file and it can be easily found by AST of current file. AST is tree representation of source code in which all constructs are represents by tree nodes. We can traverse through that tree and analyse source code. For extracting names and keys of all declarations of a source file, first an AST of source code will be created, next we will traverse through that tree and extract keys and names of all declaration nodes in that tree.
How to store keys and names of declarations?
First I thought of implementing a database which will store keys and names together. After various change of plans currently I am using DzlFuzzyIndex
in libdazzle for storing names. And for storing keys I implemented a database IdeSimpleTable
that will store strings and an array of integers associated with it. So there will be 2 indexes one for storing names and other for storing keys of declarations.
Indexing Tool
I implemented a helper tool for indexing which will take input a set of files to index, cflags for those files and destination folder in which index files needs to be stored. This will take each file from input, extract names and declarations by traversing AST of that source file and store that in DzlFuzzyIndex
and IdeSimpleTable
indexes respectively. After indexing this will create 2 files one to store names and other to store keys in destination directory.
Indexer Plugin
For indexing I implemented a plugin which will create index of source code using above tool and store that in cache folder. This will maintain a separate index for every directory. The reason behind this is if we have a single index for whole project whenever there is single change in project then entire source code needs to be reindexed. This plugin will browse through working directory and indexes a directory only if either index is not there for that directory or index is older than files in that directory. For indexing a directory, it will give list of files in that directory, cflags of those files and destination folder to store index to above tool. After indexing of project is done plugin will load all indexes and be ready for taking queries and process them.
Here is the current implementation of both indexing tool and indexer plugin. Next I will use this index to implement Global Search and Jump to definition.
Why don’t you use the LiteSQL?
LikeLiked by 1 person
Thank you for your comment. LiteSQL is based on C++, GNOME Builder supports plugins in C/Python/Vala only. Also we need to implement these features in C to be able to integrate implementation into existing code.
LikeLike
Why not try and implement a langage server client once and for all?
this is what eclipse Che, vscode, and modern IDEs do.
https://github.com/Microsoft/language-server-protocol/wiki/Protocol-Implementations .
It would then work for every langage and be an actual IDE feature instead of an ad-hoc solution to a common problem?
LikeLiked by 1 person
Fred, actually that is a planned one in GNOME Builder, https://wiki.gnome.org/Apps/Builder/Roadmap. But there are lot many things involved in that.
LikeLike
Very cool project! We work with LSP (Language Server Protocol, mentioned by Fred above) a lot at Sourcegraph and would be happy to help you spin up on it.
LikeLiked by 1 person
Thank you Quinn :).
LikeLike
I agree with other commenters that LSP support would be great.
On a related note, does this Indexer support adding additional languages? Perhaps just the ability to add regexes per-extension?
LikeLiked by 1 person
MadcapJake, yes this plugin supports indexing source code in languages other that C/C++. We changed the design so that indexing other languages will be easier.
LikeLiked by 1 person
Sorry to bother you with something offtopic. I was wondering, what font and font configuration do you have setup? I like how the fonts look in the screenshot.
LikeLike
alicia, I didn’t set up any new configuration and using existing fonts. This how global search results look like in GNOME Builder this http://imgur.com/a/6YFvB.
LikeLike
OK.
But that font you are using for GTK is clearly not Cantarell.
Thanks anyway. 🙂
LikeLike