Are these features missing or not intended? Possibility in the future?

[emrakyz]

I aimed to replace sed with sd but I guess, this is not the main idea of it.

This post is meant for the community. So, maybe some of these features will be requested by a considerable amount of people, so they will be considered to be implemented in the future.

Or, maybe the developers would recommend better alternatives, different ways we don't know about.

I am assuming that sed was developed with an aim that it would have been a scripting language on its own. You can write multi-line sed commands in a single session having conditionals, loops and similar processes. I am not completely sure if some of these are possible on sd or if you do have any plans to make at least some of these possible.

I am assuming that the syntax for sd where we split pattern and replacement in two different blocks with quotes, makes most of these impossible or harder. Or not? Please correct me if I am wrong.

I think, there would be a lot more similar or non-similar examples where sd is lacking. I will only give some examples since I could only encounter some of them trying to use it.

First Example

sed 's/\}, /\},\n	/g
	s/, /,\n	/
	s/ }/\n}/
	s/,\s*pages=/,\n\tpages=/' |
sed '1s/^ *//
	1s/[0-9]*\([0-9]\{2\}\)/\1/
	1s/_//
	1s/.*/\L&/
	s/.*=/\L&/
	s/=/ = /'

• The above command requires 10 different sd commands. Therefore, it becomes a lot slower.
• It also has some parts to make the match, lowercased. I am also assuming, this is not possible.

Second Example

It is the same for simpler commands:
sed -E '1d;$d; s/^ *([0-9]+).*\s{2,}(.+)$/\1 \2/'

• Chaining the above command with multiple sd commands or other commands such as head or tail make it a lot slower and less minimal than sed.
• In this case, we delete the first and the last line (I think this is a very common task).

Third Example

sed -E '
/Emerging/,/Completed/{/^$/d;}
/Always study the list/,+11d'
[other ones]'

• This defines a range, and it deletes the empty lines between Emerging and Completed.
  The other command also does a match, and then operate on the match to delete 11 next lines after the match.

Fourth Example

sed -e :a -e '/^\n*$/N; /^\n$/D; ta' "logfile.txt"

• We introduce a label named a. Labels are used for conditional matching and loops.
• N; appends the next line of input to the pattern space (a kind of working buffer).
• D; Deletes the first line in the pattern space and starts a new cycle (not processing the remaining commands).
• t is the conditional branch command. It tells sed to jump to a label if a previous substitution was successful.
• a is the label where the sed will jump if the substitution successful.

In short, this command removes consecutive blank lines, leaving only a single blank line between blocks of text. First match looks for an empty line. If the second match is also true, then it deletes the first match.

Fifth Example

sed -n '/0\s*c12a7328-f81f-11d2-ba4b-00a0c93ec93b/ {s|^[^ ]*|/dev/&|; s| .*||p; q}'

• Suppresses the automatic printing for everything with -n.
• We match a specific UUID (specifically the EFI Partition). Then, we only do the process on the matching line.
• Then we define multiple replacements on the same command.
• We print the specific line and quit.
• This is used with an lsblk command in order to find all partitions, filter the EFI partition, append /dev/ prefix on it. Therefore we can find the boot partition successfully with a single command.

Sixth Example

sed -i '|^nvidia/'"${GPU_CODE}"'|!d' "linux-firmware-"*

• Delete every lines that doesn't match the above pattern with !d on the Linux Firmware configuration.
• I guess this can be done with sd in a way where we match the lines completely with regex using [^pattern] and replace the result with nothing.

Seventh Example

sed -i /^FFLAGS/ a\RUSTFLAGS="-C debuginfo=0 -C codegen-units=1 -C target-cpu=native -C opt-level=3"'

• a\ Append the RUSTFLAGS variable just under the FFLAGS variable.
• I guess this can also be done by matching FFLAGS line completely with regex, then use it on the replacement with $1\nRUSTFLAGS...

Eighth Example

sed -n 's|^|/dev/|; 2p'

• Add /dev/ prefix and print the second line.

[chmln]

Hey @emrakyz thanks for taking the time to write this all out.

sd was created out of my personal frustrations with sed back then when it came to simple find and replace tasks. sed of course is much more powerful as you've noticed, having support for appends, deletions, etc. I think having a DSL like sed does would go against the simplicity of this tool, but I would be open to supporting basic things like line deletions (e.g. delete a range of lines that match a pair of regexes) with via flags for instance. The most important thing is to keep things simple and approachable, so that you don't need to look up stack overflow on how to use this command line utility. We are very much open to contributions, proposals, and discussions like this

[emrakyz]

Thanks for your answer!

I understand that the actual aim is to use this tool for simple tasks and in fact I already do that. Sometimes, I also use it in scripts for simple tasks since it's faster for simple purposes. It really makes a difference especially since we can get rid of forward slashes and escaping.

At first, I was discovering Rust based tools and found that ripgrep and fd can totally replace their counterparts but I guess they are written with the aim of rewriting their equivalent programs; not like this project. It's understandable since they are already simpler compared to sed.

I agree that the features such as append, conditionals, labels or loops that make sed behave as a language in itself are totally overkill for this project.

But I still think that some of the features can still be discussed. I think they won't make the program overly complex for the users since they can already use the current functionality.

These are just my observations and I would like to have your opinion; or other people's comments.

The features I think that are suitable for this project are:

1. Command chaining: We can use a simple flag between different replacements in order to prevent subsequent executions with pipes: sd -e 'find' 'replace' -e 'find2' 'replace2'

2. We can delete lines, ranges by numbers or matching regexes or placements such as the last line ($) using flags or similar simpler approaches without changing the current functionality or over-complicating the current usage.

3. I also think that matching specific lines with regex and do the task on those lines, is also a basic functionality but I would like to hear your opinion. For example can't we do something like this: sd --match (or -m) 'important_line' -e 'find' 'replace'

4. Reverse match seems possible and simple too.

Thanks a lot for creating this tool and I hope it doesn't die.

[stellarpower]

Personally the standard usecase for sd usually fits me - but the main other thing I will want to do is delete lines - e.g. filtering out the crap from a logfile.

I think a syntax like
sd --delete <SearchPattern>
would be quite a simple syntax compared to
sd <SearchPattern> <ReplacePattern>
, would you agree? And it should be quite clear what is intended, with two patterns for replacements, just one for deletions with the specified flag.

Another possible option - if repeated invocations slows things down, would a simple "script"-style input option work for in effect repeating invocations of sed, except on a file-by-file basis?

I.e.

./Script:

hello world
--someFlags world banana

$ sd --script ./Script

is the same as

$ sd hello world  |
  sd --someFlags world banana 

but more performant if operated through e.g. xargs or something as only one launch of the binary is required.