19 questions from the last 30 days
0
votes
1
answer
214
views
Bitnami Kafka Container [closed]
I’m trying to deploy Kafka using the Bitnami image(all versions but any case I get 'not found'):
docker pull bitnami/kafka:latest
But I get:
Status 404: failed to resolve reference "docker.io/...
0
votes
2
answers
66
views
What are safe patterns for Service A to consume data from Service B without direct DB access? [closed]
I’m looking for practical ways to let Service A get the data it needs from Service B without querying Service B’s database directly. I’m not very experienced with distributed systems yet, so I want to ...
Advice
1
vote
1
replies
56
views
Spring boot Kafka batching for all partitions
I have Spring Boot with Kafka batching and a topic with 3 partitions. For example, I want to have 1 batch with 100 records continuously if I have a lot of messages in the topic, but if I have a low ...
1
vote
1
answer
81
views
Apache Camel not propagating traceparent in Open Telemetry
I have a Camel, spring boot app that reads a message from a kafka topic. I have a spring boot test that places a message on an embedded kafka broker and add an open teleletry 'traceparent' header, ...
0
votes
0
answers
94
views
Filebeat v9.2.2 and Kafka v4.0.0 with SASL_SSL
I'm trying to set up secure log export from Filebit to Kafka and I'm running into configuration issues. Based on the documentation examples and field descriptions, I created the following configuratio,...
-1
votes
1
answer
65
views
Kafka Connect BigQuery Sink Connector continues processing despite faulty messages, DLQ not configured [closed]
I’m using a Kafka Connect BigQuery Sink Connector. Some messages fail to insert into BigQuery (for example, value is null or not a valid float), but the connector continues processing without stopping....
1
vote
1
answer
149
views
Kafka error in docker-compose: Missing required configuration "process.roles" which has no default value
I'm doing a tutorial that is quite old and the given docker compose file is referencing a Kafka image that is no longer available (bitnami/kafka:3.6). This caused docker-compose to fail:
kafka Error {&...
0
votes
1
answer
62
views
Kafka broker down because all log dirs have failed [closed]
Running Confluent Kafka 7.9.0 on Ubuntu kernel 6.8.0 in AWS EC2. Zookeeper is already up.
$ sudo systemctl restart confluent-server
$ sudo journalctl -u confluent-server.service -e
$ ... removing ...
0
votes
1
answer
58
views
Spring Kafka consumer stops consuming after 1–2 days with ExpiringCredentialRefreshingLogin logs
I have a Spring Kafka application with a single consumer. The Kafka client authentication is configured using SASL/OAUTHBEARER over SSL, as shown below:
authProps.put(SaslConfigs.SASL_MECHANISM,...
3
votes
1
answer
35
views
Kafka REST proxy: Produce message with union type
I am using Kafka, SchemaRegistry, KafkaUI and Kafka REST proxy in Docker container. I am now trying to use the REST proxy now more for test purposes, but as soon the schema is a bit more than trivial, ...
-1
votes
0
answers
51
views
Getting error about STRUCT data type while trying to configure debezium Oracle connectors [closed]
I am getting the below error while trying to replicate data from source to destination oracle server.
For all the tables, the replication of data is doing fine. But for the below tables I am getting ...
0
votes
0
answers
50
views
Kafka Streams 4.x Java 11 Compatibility
The Kafka documentation states that Kafka 4.x supports Java 11 for clients, including Kafka Streams, but not for brokers, which require Java 17.
I'm running a Kafka Streams Scala 4.x consumer. ...
-2
votes
0
answers
50
views
Kafka Connect Refuse to Flink SQL [closed]
I want to process data through flink SQL, here is my code
CREATE TABLE ohlc_source (
screener STRING,
symbol STRING,
`open` DOUBLE,
`high` DOUBLE,
`low` DOUBLE,
`close` DOUBLE,
`volume` ...
0
votes
1
answer
62
views
Spring-Boot Kafka Property-File Configuration
We use Kafka in Spring-Boot 3.4 and we want to upgrade to 4.0
We have some consumers with a Kafka-Listerner annotation. The whole config is in the application yaml-file. We just have the annotation ...
Advice
0
votes
1
replies
44
views
Are Kafka Heartbeat messages running on independent thread or are they linked to consumer poll and commit?
Is Kafka consumer heartbeat message same as consumer poll and commit? Or does it run on a separate thread and keeps sending heartbeat to Kafka broker as long as the consumer JVM is responding?
The ...