Transform input of consult-grep and consult-find commands #380
Conversation
minad
changed the title
minad
force-pushed
the
transform-grep
branch
13 times, most recently
from
3c80fef to
ebcfff1
Compare
|
It's tempting to reuse |
|
Yes, it is! I also had that planned originally but then somehow moved away from it because of the more complicated translations. If you look into the code I have some basic infrastructure there (very basic!). Maybe I should cut this down even more and focus on the Orderless integration. Probably one needs pcre2el to make this robust. |
|
Does pcre2el also have el2pcre functions? |
|
Yes, both directions. This is quite nice since then the users can configure the style as they like :) |
|
Maybe a simpler converter would suffice for Orderless as long as you don't write complicated custom regexps yourself? |
I think that's right. The regexp features used by the matching styles that come with orderless are not that fancy. Grouping without capturing, word boundaries, maybe a little more. |
minad
force-pushed
the
transform-grep
branch
3 times, most recently
from
21fdeb1 to
2539fd7
Compare
|
I refactored everything a bit, which should allow us to integrate orderless. Now we have these regexp functions:
Lines 582 to 609 in 2539fd7 In order to support Orderless it is only necessary to replace |
minad
force-pushed
the
transform-grep
branch
3 times, most recently
from
9f1d297 to
00e3742
Compare
The input string is transformed before passing it to the backend. Spaces are replaced by `.*`. Furthermore the input is interpreted as Emacs regular expression and transformed to a Perl-compatible regular expression (PCRE), by a handful of basic substitutions. The matching substrings are not obtained anymore from the backend by looking for ANSI escape sequences. Instead the input regular expression is used to heuristically apply highlighting. Advantages: - No parsing of ANSI escape sequences, which is fragile and inefficient. - More flexibility, input string can be transformed, e.g., "def consult" -> "def.*consult". - Highlighting becomes possible for backends which don't return highlights, e.g., find. Disadvantages: - Highlighting may be incorrect. In practice this is rare, since most search inputs are simple regular expressions.
|
Just to make sure I understand this, is the intended way to configure consult-grep/ripgrep/find through Example: (consult-customize
consult-ripgrep-config
:args "rg --line-buffered --color=never --max-columns=1000 --no-heading\
--line-number --follow --hidden --search-zip --smart-case .")
Last question, would you think it would be a good idea to make |
No you configure the commands by modifying the
Yes, this is a known issue, see #317. Unfortunately I don't have the possibility to test or fix this. But if someone implements a proper fix I would appreciate it. Furthermore I suggest to report this as an upstream bug to emacs-devel. This is a bug in the process API, the API abstraction is broken.
It doesn't sound like a bad idea at first. I also considered this. Downside is that we have to require the grep library then, which is not good for lazy loading. The |
|
@oantolin Do you have an idea how the issue could be fixed properly at the core? It doesn't make sense to me why this problem exists at all. |
Understood, therefore
After thinking about it I agree with you it's better to keep it separated. Also those options are made to be used with |
Yes, but I am currently overhauling this again. I am not yet satisfied with the current design and it is still lacking flexibility. The goal is to have the ability to plug in orderless as a pattern transformer, such that you can search via ripgrep with the usual orderless patterns. |
|
Could you add a comment here on whether/how you have added orderless support for grep/ripgrep commands. Please give a complete example as I find the documentation on how to send arguments to the grep commands very hard to follow. For example, give an example where a multi-word pattern is searched regardless of order but only within a single, specific file. Thanks. (Just starting with consult and the grep functionality is really great. Would like to be able to search over either multiple directories or within a specific list of files too). |
|
This still does not supply a precise example of the syntax to specify for the initial input args. Is that fully documented anywhere with specific examples? It would really help if you just gave one detailed example wherever you mention complex syntax and then explain that example rather than just listing it. For example, if I say use this: "#expr -- -g *.c#filter", no one will no what that means. But if you break it down and explain each part, then it will be clear. Thanks. |
|
To make it even simpler, I know how to write a grep command like: grep -n my-expr file1 file2. What is the exact expression to send those args to consult-grep and consult-ripgrep (or their associated commands)? |
I tried that and it errored out since 'input' is not known within the lambda expression (using latest consult and orderless packages). This sort of runs but only matches to the first character typed and only after removing the leading # async symbol: (defun consult--orderless-regexp-compiler (input type &rest _config)
(setq input (orderless-pattern-compiler input))
(cons
(mapcar (lambda (r) (consult--convert-regexp r type)) input)
`(lambda (str) (orderless--highlight ',input str))))
;; OPTION 1: Activate globally for all consult-grep/ripgrep/find/...
(setq consult--regexp-compiler #'consult--orderless-regexp-compiler) |
|
@rswgnu I think you want to do something which is not directly supported by consult-grep (but you can of course configure and adjust it). You probably want to adjust everything to your Hyperbole ways but that's not what most users necessarily want. ;) I suggest you try Consult as is for a while. Then you will see how everything fits together. Also you should read the docs completely. Most of the information you asked for should be there. Since you are an expert you could also skim the wiki and the source for additional tweaks. |
|
Hi Daniel:
I have read through a great deal of the documentation and have not been
able to solve this. It is not a Hyperbole-specific thing. I have seen
many requests for how to specify a set of files (with full or relative
paths) for consult-grep type commands to utilize and none of the proposed
solutions work with the current code.
If consult supports such a command, please show the needed inputs. If not,
I would suggest there is definitely demand. It seems like such a basic use
case. Typical examples are grepping over files that only match a glob
pattern, like *foo*.el or recursively grepping over multiple directories
potentially with a file glob as well.
In the meantime, I am utilizing consult's default behavior but this is
limiting my use.
…On Mon, Feb 20, 2023 at 2:03 AM Daniel Mendler ***@***.***> wrote:
@rswgnu <https://github.com/rswgnu> I think you want to do something
which is not directly supported by consult-grep (but you can of course
configure and adjust it). You probably want to adjust everything to your
Hyperbole ways but that's not what most users necessarily want. ;)
I suggest you try Consult as is for a while. Then you will see how
everything fits together. Also you should read the docs completely. Most of
the information you asked for should be there. Since you are an expert you
could also skim the wiki and the source for additional tweaks.
—
Reply to this email directly, view it on GitHub
<#380 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE5WPD7ASBWCW54N6QML7HDWYMJMNANCNFSM5BQ3PG7Q>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
On 2/20/23 08:17, Robert Weiner wrote:
I have seen many requests for how to specify a set of files (with full or relative
paths) for consult-grep type commands to utilize and none of the proposed
solutions work with the current code.
Then could try if you can adapt any of the existing dysfunctional
solutions, such that it works again.
If not, I would suggest there is definitely demand. It seems like such a basic use
case. Typical examples are grepping over files that only match a glob
pattern, like *foo*.el or recursively grepping over multiple directories
potentially with a file glob as well.
Sure there is always need for more commands. But we still have to draw a
line somewhere and rather provide building blocks which can then be
adapted instead of adding more and more specialized commands. How should
such the file search command look like exactly? Can you provide a
prototype? How would the workflow look like? First select some files,
then grep or first provide a wildcard pattern, then grep?
|
|
On Mon, Feb 20, 2023 at 10:35 AM Daniel Mendler ***@***.***>
wrote:
On 2/20/23 08:17, Robert Weiner wrote:
> I have seen many requests for how to specify a set of files (with full
or relative
> paths) for consult-grep type commands to utilize and none of the proposed
> solutions work with the current code.
How would the workflow look like? First select some files,
then grep or first provide a wildcard pattern, then grep?
The grep commands would take an extra argument that could be a list of
paths with glob wildcards. The Emacs wildcard expander is first used to
expand the wildcards. Then for each path that is a directory, the grep is
done like you do it now with a single directory. For each path that is a
file, it is added to the end of the grep command and simply grepped over.
If you make such updates, then we can be assured that it will be a part of
consult going forward.
Makes sense?
Message ID: ***@***.***>
… |
|
On 2/20/23 16:49, Robert Weiner wrote:
The grep commands would take an extra argument that could be a list of
paths with glob wildcards. The Emacs wildcard expander is first used to
expand the wildcards. Then for each path that is a directory, the grep is
done like you do it now with a single directory. For each path that is a
file, it is added to the end of the grep command and simply grepped over.
If you make such updates, then we can be assured that it will be a part of
consult going forward.
Makes sense. Not sure how difficult it is to implement it. But if you
want to work on it, go ahead. I don't see myself using this since I am
happy with the existing Consult commands. I like that I don't have to
select multiple directories. I mostly use consult-ripgrep to search
recursively through projects.
|
|
Hi Daniel:
I'll work on it when I get time on the weekend and will submit a PR when
ready.
On Mon, Feb 20, 2023 at 11:10 AM Daniel Mendler ***@***.***>
wrote:
… On 2/20/23 16:49, Robert Weiner wrote:
> The grep commands would take an extra argument that could be a list of
> paths with glob wildcards. The Emacs wildcard expander is first used to
> expand the wildcards. Then for each path that is a directory, the grep is
> done like you do it now with a single directory. For each path that is a
> file, it is added to the end of the grep command and simply grepped over.
> If you make such updates, then we can be assured that it will be a part
of
> consult going forward.
Makes sense. Not sure how difficult it is to implement it. But if you
want to work on it, go ahead. I don't see myself using this since I am
happy with the existing Consult commands. I like that I don't have to
select multiple directories. I mostly use consult-ripgrep to search
recursively through projects.
—
Reply to this email directly, view it on GitHub
<#380 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE5WPD5YHYTXY3W5NWLCPV3WYOJQHANCNFSM5BQ3PG7Q>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
The input string is transformed before passing it to the backend. Spaces are
replaced by
.*. Furthermore the input is interpreted as Emacs regularexpression and transformed to a Perl-compatible regular expression (PCRE), by a
handful of basic substitutions. The matching substrings are not obtained anymore
from the backend by looking for ANSI escape sequences. Instead the input regular
expression is used to heuristically apply highlighting.
Advantages:
Disadvantages:
are simple regular expressions.