In a recent preprint, I investigated $$S_p(x):=\prod_{k=1}^{(p-1)/2}(x-e^{2\pi ik^2/p}),$$ where $p$ is an odd prime and $x$ is a root of unity.
Motivated by Question 337879 and Question 338325, here I pose my conjecture on $$s_p:=S_p(e^{2\pi i/12})=\prod_{k=1}^{(p-1)/2}(e^{2\pi i/12}-e^{2\pi i k^2/p})$$ for primes $p>3$.
Conjecture. Let $p>3$ be a prime.
(i) If $p\equiv13\pmod{24}$, then $$s_p=i(-1)^{\frac{p-5}8+|\{1\le k<\frac p4:\ (\frac kp)=-1\}|}(x_p\sqrt3-y_p\sqrt p),$$ where $(\frac kp)$ is the Legendre symbol, and $(x_p,y_p)$ is the least positive integer solution to the diophantine equation $3x^2+1=py^2$.
(ii) When $p\equiv19\pmod{24}$, we may write $p=(4x)^2+3y^2$ with $x,y\in\mathbb Z$, and we have $$s_p=(-1)^{(p-19)/24+x}(1+i)\frac{1+\sqrt3}2.$$
(iii) If $p\equiv1,7\pmod{24}$, then $$(-1)^{\lfloor \frac{h(-p)}2\rfloor+|\{1\le k<\frac p{12}:\ (\frac kp)=1\}|} e^{2\pi i(p-1)/48}s_p>0,$$ where $h(-p)$ is the class number of the imaginary quadratic field $\mathbb Q(\sqrt {-p})$.
(iv) If $p\equiv5\pmod6$ but $p\not\equiv23\pmod{24}$, then $$(-1)^{\lfloor \frac{h(-p)}2\rfloor+|\{1\le k<\frac p{12}:\ (\frac kp)=1\}|} e^{2\pi i5(p-1)/48}s_p>0.$$ When $p\equiv 23\pmod{24}$, we have $$(-1)^{\lfloor \frac{h(-p)}2\rfloor+|\{1\le k<\frac p{12}:\ (\frac kp)=1\}|} e^{2\pi i5(p-1)/48}s_p<0.$$
I have checked my above conjecture numerically.
QUESTION. How to prove the conjecture? How to determine the exact value of $s_p$ for a general prime $p>3$ with $p\not\equiv13,19\pmod{24}$?
Your comments are welcome!
