Fix trailing ) from interfering with extraction in Clojure keywords

[eneroth]

Summary

In a form like,

(if condition :bg-white :bg-black)

:bg-black will fail to extract, while :bg-white is extracted as expected. This PR fixes this case, implements more comprehensive candidate filtering, and supersedes a previous PR.

Having recently submitted a PR for handling another special case with Clojure keywords (the presence of : inside of keywords), I thought it best to invert the previous strategy: Instead of handling special cases one by one, consume keywords according to the Clojure reader spec. Consume nothing else, other than strings.

Because of this, this PR is a tad more invasive rather than additive, for which I apologize. The strategy is this:

• Strings begin with a " and ends with an unescaped ". Consume everything between these delimiters (existing case).
• Keywords begin with :, and end with whitespace, or one out of a small set of specific reserved characters. Everything else is a valid character in a keyword. Consume everything between these delimiters, and apply the class splitting previously contained in the outer loop. My previous special case handling of : inside of keywords in Extract candidates with variants in Clojure/ClojureScript keywords #18338 is now redundant (and is removed), as this is a more general solution.
• Discard everything else.

I'm hoping that a strategy that is based on Clojure's definition of strings and keywords will pre-empt any further issues with edge cases.

Closes #18344.

Test plan

• Added failing tests.
• cargo test -> failure
• Added fix
• cargo test -> success

[thecrypticace]

@eneroth Can you see if the checkbox to allow maintainers to edit the PR is checked? I've got a few tweaks I want to make.

[thecrypticace]

Also could use a changelog entry

[eneroth]

Sorry @thecrypticace, from what I can see on Github docs, there's supposed to be a checkbox for me to check here:

But I can't see any. In lieu of that, rebased on main and amended with your changes, and added a change log entry.

[thecrypticace]

Huh that's super weird. I wonder why it's missing. Thanks for making the changes!

[thecrypticace]

Ah weird | b'#' being gone makes the tests fail. I guess we can keep it in for the time being then. 🤔

edit: I know why. It's a bit annoying. We need to read whole keywords then split them into utilities. I wonder if this means we need an expanded character set no matter what.

Can do that as a followup bug fix if we turn out to need that.

[thecrypticace]

I'm gonna go head and merge this so I can get it in. Will fix the failing # test in a followup commit.

[eneroth]

@thecrypticace Ah yeah, because of the syntax <elem>#<id>.<class-1>.<class-2>…<class-n>. This is unfortunately a requirement for Clojure for the Hiccup DSL to work.

[thecrypticace]

Yeah, it makes sense. We just can't be quite as selective about certain characters during the pre-processing stage which is fine.

[yannvanhalewyn]

Ironically in v4.1.12 extraction of classes in symbol lists were broken. I'm not 💯 sure it's due to this PR but something to keep in mind.

[:div {:class '[class-1 class-2
                class-3 class-4]}]

[eneroth]

@yannvanhalewyn Goodness, I wasn't even aware that classes as symbols were a thing, sorry if that was due to my intervention.

Doesn't symbols in that manner make conditional logic etc. tricky?

[yannvanhalewyn]

@eneroth no worries it happens :). We've pinned back to 4.1.11 until this works again.

It composes pretty well for conditional logic:

(def hl-class-names '[ring ring-blue-500])

[:div
 {:class (cond-> '[input w-full]
           textarea? (conj 'textarea)
           (seq errors) (concat '[border-red-500 bg-red-100])
           highlight? (concat hl-class-names))}]

Another benefit is it allows splitting long classlists over multiple lines, and introducing logical grouping:

[:div
 {:class '[h-100 lg:h-200 max-w-32 mx-auto py-60
           flex flex-col justify-end items-center
           lg:flex-row lg:justify-between
           bg-cover bg-center bg-no-repeat rounded-3xl overflow-hidden
           font-semibold text-gray-900]}]

Note: slashes make them invalid tokens, those need to be explicit strings: '[p-32 "text-black/50"]