2,233 questions
-3
votes
0
answers
43
views
Avoiding counter update contention under high write throughput [closed]
We maintain multiple counters where each incoming request increments or decrements one or more counters. These counters are bounded by a max value, once that is reached, we reject the requests.
...
- 34.5k
0
votes
0
answers
28
views
Why does MixCoord keep routing requests to stale QueryNodes after a Kubernetes node reboot in Milvus?
MixCoord keeps routing requests to non-existent QueryNodes after a Kubernetes worker node reboot in Milvus
I’m running a Milvus 2.5.x cluster on Kubernetes, where each worker node hosts a full set of ...
0
votes
1
answer
55
views
Efficient distributed sequence
I've got a lot of applications, each one have her own log. I'm typically in a distributed logs problematic.
To resolve this problematic, I'm using :
Brave / Spring Sleuth to generate trace and span ...
- 42.5k
3
votes
3
answers
3k
views
How to implement "Distributed cache clearing"?
We have multiple instances of Ofbiz/Opentaps running. All the instances talk to the same database. There are many tables that are rarely updated hence they are cached and all the instances maintain ...
- 81.1k
2
votes
1
answer
628
views
Celery CLI - cannot cancel consumer on particular worker on particular node
Using flower I can cancel or add consumer for particular worker on particular node.
However using CLI, I fail and getting this error:
Error: No nodes replied within time constraint.
This is the ...
- 14.6k
0
votes
0
answers
52
views
Distributed Tensorflow with mulitple GPUS training MNIST with Optuna is stuck when training
I created a 5 GPU Cluster using three nodes/machines locally using the tensorflow.distributed.MultiWorkerMirrored Strategy. One machine has the Apple M1 Pro Metals GPU, the other two nodes has NVIDIA ...
24
votes
7
answers
82k
views
Search/Find a file and file content in Hadoop
I am currently working on a project using Hadoop DFS.
I notice there is no search or find command in Hadoop Shell. Is there a way to search and find a file (e.g. testfile.doc) in Hadoop DFS?
Does ...
- 1,313
32
votes
2
answers
21k
views
What are the essentials of real-time distributed systems?
I am getting my foot into contracting and have had today my first round interview for a contractor position. I have passed it however I was told - being mainly a UI developer - I only covered the ...
32
votes
12
answers
11k
views
How to build large applications [closed]
I think I've become quite good at the basics of programming (for a variety of languages). I can write a good line of code. I can write a good method. I can write a good class. I can write a good ...
- 21.3k
1
vote
3
answers
882
views
Can Sloppy Quorum guarantee strong read consistency?
In the book "Designing Data-Intensive Applications. The Big Ideas Behind Reliable, Scalable and Maintainable Systems", we can read regarding Sloppy Quorum :
However, this means that even ...
- 75
12
votes
2
answers
5k
views
Why distributed cache is faster than database?
Assume data is stored the same in database and in distributed cache (.i.e. no join needed), is it still relevant that distributed cache much faster than accessing database directly?
As far as I ...
- 1
2
votes
2
answers
230
views
Client request failure in raft
Imagine a 3 node raft cluster. Each node is in sync has log [1,2,3] and entry 3 is committed by the leader.
Now leader receives an entry 4 but fails to commit it because of unreliable network and ...
- 49k
120
votes
6
answers
105k
views
In Apache Kafka why can't there be more consumer instances than partitions?
I'm learning about Kafka, reading the introduction section here
https://kafka.apache.org/documentation.html#introduction
specifically the portion about Consumers. In the second to last paragraph in ...
- 15.3k
0
votes
0
answers
44
views
How does clusters work in TensorFlow in the parameterServerStrategy?
I don't seem to understand how clusters work in the parameterServerStrategy in TensorFlow, and I need some clarifications.
I have read this tutorial, but they don't mention or explain clearly how to ...
0
votes
0
answers
82
views
Low CIFAR-10 Accuracy (60%) in Decentralized Federated Learning (DFL) - Seeking Improvement
I implemented an algorithm in a Decentralized Federated Learning (DFL) environment. When I experimented with MNIST and Fashion-MNIST, I achieved an accuracy of 80–90%. However, when testing with CIFAR-...
