Backticked text in docstrings is not rendered as monospace

[sarna]

See for example spork/argparse - in Usage we have properly monospaced text, but just below that, inside the docstring, there's raw text like:

Parse `(dyn :args)` according to options. If the arguments are incorrect,
will return nil and print usage information.

I don't know if docstrings support markdown (or some other markup), but at least in the repl when you run (doc argparse/argparse) the backticked parts are rendered in a different color, so I think that they should get special treatment on the website too.

[sogaiu]

Don't have a solution to this ATM, but below are some results of some investigation.

It looks to me like the portion in question is the result of processing this line:

@api-docs("../../spork" "argparse")

AFAIU, @api-docs is defined in mendoza here:

(defn api-docs
  "Generate docs for a given module. Returns a node."
  [module &opt prefix]
  (def env (if (string? module) (require module) module))
  (seq [[k entry]
        :in (sort (pairs env))
        :when (symbol? k)
        :when (or (not prefix) (string/has-prefix? prefix k))
        :when (and (get entry :doc) (not (get entry :private)))]
    (emit-item k entry)))

It looks like emit-item might be relevant and it's definition has the following bits:

(defn- emit-item
  "Generate documentation for one entry."
  [key env-entry]
  (let [{:macro macro
         :value val
         :ref ref
         :source-map sm
         :doc docstring} env-entry

        doc2 (or docstring "")
        doc-dom (peg/match docstring-peg doc2)]
    {:tag "div" "class" "docstring" :content
     [{:tag "div" "class" "binding-wrap"
       :content [{:tag "span" "class" "binding" :content {:tag "a" "id" key :content (string key)}}
                 {:tag "span" "class" "binding-type" :content full-binding-type}
                 source-ref]}
      doc-dom]}))

My guess is that env-entry is what one might get back from doing something like (dyn 'loop) or (get root-env 'loop), i.e. the result may be a table with keys such as :doc, :macro, :value, etc.

The value associated with the :doc key is a docstring (still has backticks and friends) with a "preamble" (often a function signature and some newlines).

Roughly, the docstring that's obtained is processed by (peg/match docstring-peg ...) and the PEG referred to there is a compiled version of docstring-peg-source defined here as:

(def- docstring-peg-source
  "Convert a docstring into a dom node."
  ~{:ws (set " \t\r\n\0\f")
    :nn (* "\n" (any (set " \t\r\f")) "\n")
    :funcdef (* (any :ws)
                (/ '(* "(" (any (if-not ")" 1)) ")")
                   ,|(codeblock "janet" $))
                :nn)
    :br (* :nn (constant {:tag "br" :no-close true}))
    :li (* (+ (* (+ "-" "*") :s+) "\t") (/ '(any (if-not "\n" 1)) ,|{:tag "li" :content $}))
    :ul (* (some :li) (+ "\n" -1))
    :sent '(some (if-not "\n" 1))
    :main (* (? :funcdef) (/ (group (any (+ :ul :sent :br "\n"))) ,|{:tag "p" :content $}))})

(def- docstring-peg (peg/compile docstring-peg-source))

So to summarize, regarding:

at least in the repl when you run (doc argparse/argparse) the backticked parts are rendered in a different color, so I think that they should get special treatment on the website too.

one way to achieve this might be to modify the aforementioned PEG in mendoza.

---

Regarding what is supported in docstrings, see this issue.

On a related note, in that issue there is mention of @pyrmont's paused PR. If some of that were implemented, possibly it might help for issues like this one.

[sogaiu]

Oh, regarding:

at least in the repl when you run (doc argparse/argparse) the backticked parts are rendered in a different color

AFAIU, this works because doc-format has some "smarts" for handling color, either via its optional colorize parameter or via the dynamic variable *doc-color*.

---

IIRC, part of @pyrmont's PR is about exposing the part of doc-format that creates a parse tree via a function named doc-parse (and then having doc-format call that).

I think getting access to the parse tree would allow us to improve producing content "for the web":

To display these docstrings, it renders the docstring as plaintext and then wraps these in HTML’s <pre> tag. While this is fine for most desktop environments, on smaller-width screens (like phones), wrapping ‘breaks’ if the width of the display is less than the width at which the docstring was rendered to plaintext. [...] if the parse tree were accessible, we think it would be relatively trivial to transform this into the Mendoza template language so that the text included more semantic information in the markup (e.g. lists and code blocks).

I think pyrmont made a fair bit of progress on this but in this case "life didn't find a way" [1] unfortunately.

(If the PR were revisited at some point, I think it would be ever so slightly easier because the underline feature has been removed from doc-format.)

---

[1] We didn't pray enough to Jeff Goldblum I guess :P