yaml-pro-beautify

[holocronweaver]

I am looking for the YAML equivalent of json-mode-beautify, which auto-formats a JSON buffer.

Does anything like yaml-pro-beautify exist?

[rodrigo-morales-1]

@holocronweaver I suppose you are you referring to this function json-mode-beautify. If so, I'm not aware of anything like this in yaml-pro.el.

[holocronweaver]

Thanks @rdrg109, that is indeed the correct function. I see it relies on the built-in Emacs json-pretty-print from the json module.

Would adding something similar for YAML be straightforward? I suspect not unless Emacs adds a similar yaml module.

[dcunited001]

Thanks @rdrg109, that is indeed the correct function. I see it relies on the built-in Emacs json-pretty-print from the json module.

Would adding something similar for YAML be straightforward? I suspect not unless Emacs adds a similar yaml module.

It depends on what you're looking for. The tree-sitter integration in emacs allows you to specify rules for indenting various grammar nodes. See How to Get Started with Tree-Sitter. Also, take a look at python-ts-mode and clojure-ts-mode.

I'm not sure if that was the answer you were looking for. It would be better to use an external tool though. There may be problems specific to each templating system.

it may be possible to resolve Templating & LSP issues by using a secondary buffer and fuzzing anything that's supposed to turn into a string. That way, you can track the differences and bring the changes back into the original buffer... but that's insane.

Also, I think TS supports combining grammars but I just briefly looked into it and I didn't find much.

[holocronweaver]

Thanks @dcunited001 for the detailed response! I was thinking using an external tool or module would likely be best, both for performance and avoiding reinventing the wheel.

The tree sitter approach is interesting though, I wasn't aware it was added recently to Emacs. I noticed in the above linked How to Get Started article that the Combobulate package supports YAML, perhaps that can serve as a starting place?

[uqix]

I use yq (brew install yq on macOS) and shell-command-on-region to do the job:

(with-eval-after-load 'embark
  (keymap-set embark-region-map "c j y" #'my/region/convert/json/to-yaml)
  (keymap-set embark-region-map "c j j" #'my/region/convert/json/prettify)
  (keymap-set embark-region-map "c y j" #'my/region/convert/yaml/to-json)
  (keymap-set embark-region-map "c y y" #'my/region/convert/yaml/prettify)
  (keymap-set embark-region-map "c y p" #'my/region/convert/yaml/to-properties)
  (keymap-set embark-region-map "c p y" #'my/region/convert/properties/to-yaml))

(defun my/region/convert/json/to-yaml ()
  (interactive)
  (my/region/convert/by-shell-command "yq -p=json -o=yaml -"))

(defun my/region/convert/json/prettify ()
  (interactive)
  (my/region/convert/by-shell-command "yq -p=json -o=json -"))

(defun my/region/convert/yaml/to-json ()
  (interactive)
  (my/region/convert/by-shell-command "yq -p=yaml -o=json -"))

(defun my/region/convert/yaml/prettify ()
  (interactive)
  (my/region/convert/by-shell-command "yq -p=yaml -o=yaml -"))

(defun my/region/convert/yaml/to-properties ()
  (interactive)
  (my/region/convert/by-shell-command "yq -p=yaml -o=props -"))

(defun my/region/convert/properties/to-yaml ()
  (interactive)
  (my/region/convert/by-shell-command "yq -p=props -o=yaml -"))

(defun my/region/convert/by-shell-command (command)
  (shell-command-on-region
   (region-beginning)
   (region-end)
   command
   t
   t))

[holocronweaver]

Very nice @uqix! yq with your defuns works pretty decently to prettify YAML (though it sometimes handle whitespace between comment blocks in oddly inconsistent ways).

Would be great if this could be added to yaml-pro as a first pass pretty formatter, maybe with a little blurb / link for installing yq if not found at the default path.

[zkry]

Apologies for the delay in responding. That's a good idea to include yq tooling. It shouldn't be too much effort to include.

Also, I was thinking it may be possible to include pretty formatting using just treesitter. Looking at yq, it appears the things it formats are:

• no blank lines
• - lists are always intended
• indentation is always two spaces
• | stings are minimally indented.
• On a key-value pair, there is only one space after :
• JSON-like elements are onelined, no matter the length

Seems like these rules should get it most of the way.

[uqix]

Formatting by yq does have its limits, it'd be best to handle them at yaml-pro (tree-sitter) side.

[holocronweaver]

I suggest going with yq for a short term solution in yaml-pro, and looking a tree sitter long term (unless someone who is familiar with it can implement it as quickly). I suspect tree sitter will take allot more effort.

[zkry]

So I was doing some more exploring around this space and found that prettier.js does have a YAML formatter. This could be incorperated via something like https://github.com/prettier/prettier-emacs.

With that said I do think there could be value in having a light-weight formatter in this package (would run faster and you wouldn't need to depend on npm ecosystem). While I think getting to 1-to-1 parity with prettier.js would be a difficult task, I feel like it would be possible to get 80% of the value with 20% of the work.

Here are the prettier.js rules I've inferred so far:

• Formatting around ":" in block mapping
• Formatting after "-" in block sequence
• Multiple spaces reduced to one space (e.g. ["a", "b", "c"])
• Empty lines are reduced down to one empty line
• Only 2 spaces for indentation
  • Block scalars, if they don't have a indent spec should be minimally indented.
  • Multi-line strings should be minimally indented if they span multiple lines.
• Comments should indent to the elements after them
• flow_mapping past column X should be broken
• flow_sequence past column X should be broken
• single quotes should change to doulbe quotes if they don't have backslash

[zkry]

Started working on this and was able to get formatting to work pretty well. #38
It seems it would be possible to get most of the features of the prettier.js features without too much effort. The main thing left would be to break up long flow items onto multiple lines.

[holocronweaver]

Looks very promising @zkry!

I am surprised by the mix of regex and tree sitter - is that typically how tree sitter is used?

The code seems to break each aspect of formatting into separate functions - am I right that this would make it relatively easy to customize which rules are enforced and even override what they do?

[zkry]

I am surprised by the mix of regex and tree sitter - is that typically how tree sitter is used?

I haven't seen treesitter used quite this way, and I'm not sure it's the best way to do this, but it seems necessary when trying to find whitespace. I'm not aware of a good treesitter-only way of querying for specific whitespace patters. Ideally there would be a parser behind this. Yaml.el could do such a thing but it's way too slow now to be usefull as a formatter. I've got the prettier.js tests so I'll be using those to iterate on this until something reliabe comes out.

The code seems to break each aspect of formatting into separate functions - am I right that this would make it relatively easy to customize which rules are enforced and even override what they do?

This is definitely the hope. Getting into the weeds of YAML syntax, there's so many edge cases, but I hope to be able to break it into things which can be customized.

[zkry]

So the experiment was successful! On the latest master, there is now a function called yaml-pro-format-ts which will format the buffer according to a set number of rules (set in custom var yaml-pro-format-features, I recommend setting it with customize-variable). I managed to get it pretty close to what prettier.js outputs. Most of the differences come from the way tree-sitter parses.

I'll work on adding documentation and clean things up and will draft a new release. If you have the chance to try it in the meantime definitely let me know your thoughts and if there's anything that I can improve.

[holocronweaver]

Will give it a try when I'm done traveling.

BTW, since use of treesitter is an implementation detail, maybe name the function simply yaml-pro-format?