112,838 questions
0
votes
2
answers
59
views
Read files from storage account in Spark - Without using keys - Azure
I am doing local development. Wish to run spark job locally on my desktop, and access files in storage account from my spark job.
I don't have an option to use SAS tokens or access-keys for my storage ...
- 31.8k
0
votes
1
answer
28
views
bigtable spark connector not reading cell timestamp with the data
I am using bigtable spark connector to read bigtable data in scala code. I want to read the cell timestamp with the data as well. But nowhere I can find how to do it. Can someone help me on this?
I ...
- 226
0
votes
1
answer
32
views
Global Temp view shows up empty when passed from Pyspark to Scala notebook in Databricks
I have a Azure Databricks workflow which runs a Pyspark notebook which in turns calls a Scala notebook(legacy) for a list of tables. In Pyspark notebook, I save a Dataframe to a GlobalTempView and ...
- 3
0
votes
0
answers
60
views
Unable to read schema from Karapace Schema registry using Confluent client, the curl command works fine
I'm unable to read schema from Karapace Schema registry using Confluent client, the curl command works fine
here is the scala code -
def getSchemaFromRegistry(schemaRegistryUrl: String, subject: ...
- 1,093
0
votes
0
answers
67
views
How to fail Spark on wrong schema?
The data:
{"name": "name1", "id": 1}
{"name": "name2", "id": "1"}
The code:
val schema =
"""
| name ...
- 33.9k
2
votes
0
answers
63
views
SyntacticRule with optional Semantic in Scalafix?
I am working on Rules to switch from Kyo 0.19 to Kyo 1.0, is there a way to make those rules Syntactic, with some optional Symbol checks if it's possible (semanticDb is available)?
package kyo.rules
...
- 1,769
0
votes
0
answers
86
views
Google Sign-In fails silently with /login?error when using HTTPS (works on HTTP) in Spring Boot + Scala application
I am facing an issue with the Google Sign-In integration in my Scala + Spring Boot web application.
Problem:
When I try to sign in with Google using HTTPS (deployed via ngrok), I am silently ...
0
votes
2
answers
57
views
sbt plugin resolvers not adding sbtVersion, scalaVersion to artifactory url
In an sbt project, I'm adding some plugins, like
addSbtPlugin("org.typelevel" % "sbt-tpolecat" % "0.5.2")
This should resolve to an artifact url similar to this:
https://...
- 573
0
votes
1
answer
40
views
Generic way of array creation in Scala.js
I'm looking for a generic way of array creation in Scala.js.
The following code works fine in JVM and Scala Native platforms:
def newArray[A](fqcn: String, size: Int): Array[A] = {
val clazz = Class....
- 8,033
1
vote
1
answer
59
views
Referring to non-existent class while compiling to ScalaJS
I'm builing a ScalaJs & Scala3 application, when i run the npm run dev/build command I have the following error, and I'm not figuring out how to fix it:
Referring to non-existent class scala....
0
votes
2
answers
68
views
conditional map fetch efficiently
I have a map which lays down prices of commodities in different currencies
val commpricemap: Map[String , Map[String, Double]] = ???
AN example of an entry for gold is as below:
("AU" -> ...
- 9,927
0
votes
0
answers
30
views
mill debugger intellij in Xiangshan project
I'm trying to debug the XiangShan RISC-V SoC project using IntelliJ IDEA with Mill and BSP.
I've followed all recommended steps (scala, the mill build tool, and the IntelliJ debugger) but breakpoints ...
0
votes
2
answers
90
views
Spark Unit test failing maven test but pass in IntelliJ
I'm working on a Scala project using Spark (with Hive support in some tests) and running unit and integration tests via both IntelliJ and Maven Surefire.
I have a shared test session setup like this:
...
- 1,813
0
votes
0
answers
53
views
Newest Java 24 can't run Play because of class file version mismatch [duplicate]
I've created a project that ran perfectly fine 2 weeks ago with Scala and Play Framework. It worked, but now it doesn't, but the error confuses me a lot. I used Java 24 and also tested on Java 17, but ...
0
votes
1
answer
80
views
Transform a match to avoid duplicate
Is there a construct which allows to drop the repetition of the 0.0 routine below
val price: Option[Double] = ???
price match {
case Some(d) =>
if (isPriceFair(d))
d
else
...
- 9,927