Skip to main content
2,827 questions
Newest Active Bountied Unanswered
72 votes
6 answers
120k views

I setup a single node Kafka Docker container on my local machine like it is described in the Confluent documentation (steps 2-3). In addition, I also exposed Zookeeper's port 2181 and Kafka's port ...
Sasha Shpota's user avatar
  • 10.5k
34 votes
10 answers
42k views

I'm using a Kafka Source in Spark Structured Streaming to receive Confluent encoded Avro records. I intend to use Confluent Schema Registry, but the integration with spark structured streaming seems ...
Souhaib Guitouni's user avatar
  • 882
196 votes
11 answers
333k views

I send String-messages to Kafka V. 0.8 with the Java Producer API. If the message size is about 15 MB I get a MessageSizeTooLargeException. I have tried to set message.max.bytesto 40 MB, but I still ...
Sonson123's user avatar
  • 11.5k
41 votes
9 answers
103k views

I'm trying to install Kafka message queue on Windows for testing purposes (not for production). I found this article on how to install Apache Kafka 0.8 on Windows: http://janschulte.wordpress.com/...
Yves M.'s user avatar
  • 31.3k
44 votes
3 answers
26k views

What I'd like to do is this: Consume records from a numbers topic (Long's) Aggregate (count) the values for each 5 sec window Send the FINAL aggregation result to another topic My code looks like ...
odavid's user avatar
  • 649
47 votes
6 answers
52k views

I want to create a topic in Kafka (kafka_2.8.0-0.8.1.1) through java. It is working fine if I create a topic in command prompt, and If I push message through java api. But I want to create a topic ...
Jaya Ananthram's user avatar
  • 3,463
10 votes
2 answers
53k views

I have been wanting to use Apache Storm to stream from Kafka. I am more comfortable with Python, so I decided to use streamparse (https://github.com/Parsely/streamparse). The word count example is the ...
red_devil's user avatar
  • 1,049
327 votes
7 answers
109k views

Although I've come across Kafka before, I just recently realized Kafka may perhaps be used as (the basis of) a CQRS, eventstore. One of the main points that Kafka supports: Event capturing/storing, ...
Geert-Jan's user avatar
  • 19k
136 votes
1 answer
151k views

I am unclear why we need both session.timeout.ms and max.poll.interval.ms and when would we use one or the other or both? It seems like both settings indicate the upper bound on the time the ...
Deeps's user avatar
  • 2,099
55 votes
11 answers
62k views

How Can we create a topic in Kafka from the IDE using API because when I do this: bin/kafka-create-topic.sh --topic mytopic --replica 3 --zookeeper localhost:2181 I get the error: bash: bin/kafka-...
ramu's user avatar
  • 1,325
53 votes
1 answer
40k views

Traceback (most recent call last): File "//producer.py", line 1, in <module> from kafka.producer import KafkaProducer File "/usr/local/lib/python3.9/site-packages/kafka/...
Harvey's user avatar
  • 826
270 votes
27 answers
380k views

I pushed a message that was too big into a kafka message topic on my local machine, now I'm getting an error: kafka.common.InvalidMessageSizeException: invalid message size Increasing the fetch.size ...
Peter Klipfel's user avatar
  • 5,208
286 votes
3 answers
202k views

I am relatively new to Kafka. I have done a bit of experimenting with it, but a few things are unclear to me regarding consumer offset. From what I have understood so far, when a consumer starts, the ...
Asif Iqbal's user avatar
  • 4,812
40 votes
5 answers
32k views

I'm trying to find out which offsets my current High-Level consumers are working off. I use Kafka 0.8.2.1, with no "offset.storage" set in the server.properties of Kafka - which, I think, means that ...
Marina's user avatar
  • 4,084
15 votes
1 answer
13k views

I am trying to use structured streaming approach using Spark-Streaming based on DataFrame/Dataset API to load a stream of data from Kafka. I use: Spark 2.10 Kafka 0.10 spark-sql-kafka-0-10 Spark ...
Stefan Repcek's user avatar
  • 2,603

15 30 50 per page
1
2 3 4 5
…
189