Plain SQS is already roughly ordered, and will suffice for most use cases. Kinesis bills by data volume (megabytes), while SQS bills by message count - do the math for your use case.ĭon't worry about SQS FIFO unless you need the guarantees it provides around ordering.You can perform similar logic with Kinesis (it's built to have multiple consumers), but the extra development complexity and cost is typically not worthwhile unless you are dealing with very large message volumes.Basic concepts and design remains will be same for most of the messaging queue technologies irrespective of what you. Apache Kafka, ActiveMQ, RabbitMQ, ZeroMQ and SQS are few well known messaging queue technologies available in the market. If you need them to be in sync, then that won't work - you should just use a single queue, and a process to synchronously poll the queue and deliver the message to each application. Messaging queue is important component for coordinate between decoupled services and as Pub-Sub mechanism.In general, that should be ok as long as the consumers are independent - the queue acts as the buffer so no information is lost. Each consumer application (that is attached to a queue) will consume at its own rate - this means that it's possible for one or more to "fall behind".This looks like a perfect use case for SNS-SQS fanout notifications - the messages are sent to an SNS "topic", and SNS will deliver it to multiple SQS queues that are "subscribed" to that topic. I am trying to read as much as possible but would definitely appreciate if someone can help me make the right decision I would really appreciate some help on this. Helpful_Info I used this article just to understand the differences This one answers why Kenesis can be used for my scenario Link_1 This post answers the question of using multiple consumers with the Queue but not sure if it addressing the issue of same messages consumed by multiple consumers Some of the questions I went through on stackoverflow:
Primary source of information is Amazon docs and I think I am confused between all the options and overwhelmed by all the information available on the Queues but still confused about which architecture to choose I see the option of Standard SQS, SQS FIFO, (SQS + SNSTopic) & Kenesisįor the functionality that I want it seems like either (SQS + SNS Topic) or Kenesis would be the way to go.īut I also have a question regarding Standard SQS & SQS FIFO - Is it not possible for all of the consumers to get the same message if I use SQS FIFO or Standard SQS? I am not sure what Queuing architecture to use for this purpose.
I have one primary application sending messages to SQS Queue and want 4 consumer applications to consume the same message and process it however they want to
0 kommentar(er)
