Unset your PHP references!

foreach ($infs as &$inf) {
   …
}

foreach($infs as $i => $inf) {
   …
}

What’s wrong with the above code? Why after the second loop are there duplicate entries in my $infs array when I did no such thing?

The reason is because I reused the reference variable $inf. That’s a big no-no. To be safe, always unset reference variables when you’re done with them, like this:

foreach ($infs as &$inf) {

}
unset($inf);


MongoDB at Traackr: Indexing and Ad-Hoc Querying

MongoDB at Traackr: Indexing and Ad-Hoc Querying

Redis and Lua

Redis and Lua

Just Dave: How to use Traackr influencers monitor as your screen saver.

Just Dave: How to use Traackr influencers monitor as your screen saver.

8 Reasons To Choose A Startup Over A Corporate Job

8 Reasons To Choose A Startup Over A Corporate Job

RabbitMQ Msg Consumption Rates

A friend and fellow programmer asked me recently if RabbitMQ could handle sending 1000 messages per second. It’s a bit tricky to answer because message consumption rates not only rely on (1) the time it takes a message to travel over the wire from the RabbitMQ server to your client, and (2) the time it takes your code to process the message.

So, as a simple test, I created a test queue on my local RabbitMQ server and populated it with a 100,000 test messages, which have only a UUID field as such:

public class TestMsg implements Serializable {
   
    private static final long serialVersionUID = 6810143110016051212L;
   
    private String uuid = UUID.randomUUID().toString();
   
    public TestMsg() {
    }
   
    @JsonCreator
    public TestMsg(@JsonProperty("uuid") String uuid) {
        this.uuid = uuid;
    }
   
    public String getUuid() {
        return uuid;
    }
}

Likewise, my message consumer was extremely simple and only printed out the message UUID for every 1000 received, as such:

    ...
private static AtomicInteger count = new AtomicInteger(0);

public void handleMessage(final TestMsg tst) throws Exception {
int currentCount = count.incrementAndGet();
if (currentCount % 1000 == 0) {
System.out.println("[" + currentCount + "] " + tst.getUuid());
}

if (currentCount == 100000) {
System.out.println("END - " + Calendar.getInstance().getTimeInMillis());
}
}
...

Round 1:
Setup: 1 consumer -> queue
Total time: 117395(ms) ~ 118 seconds
Msg Rate: ~847 msgs/sec (100k/118) … so this does not meet our goal with just one consumer pulling from the queue

Round 2:
Setup: 2 concurrent consumers -> queue
Total time: 80462(ms) ~ 81 seconds
Msg Rate: 1234 msgs/sec (100k/81) … w00t we did it!

Round 3:
Setup: 4 concurrent consumers -> queue
Total time: 70475(ms) ~ 71 seconds
Msg Rate: 1408 msgs/sec (100k/71)


     Go-go Gadget Go!

On my laptop, I’m limited to one processor w/2 cores, so that’s why I believe I didn’t see much improvement when increasing the number of parallel consumers from 2 to 4. On beefier machines, I think it would make a great difference.

So, back to the original question, I believe this shows that RabbitMQ is capable of delivering 1000/msgs a second. Granted, I’m not dealing with any network latency on my laptop, but network latency will pose the same delays for any message broker, not just RabbitMQ.

And what you do with the message if up to your code, and will have the greatest impact on consumption rates. For instance, if it takes your code 10 milliseconds to process a message, and you have 2 concurrent message consumers, then at most the best message rate you could get would be 200 msgs/sec. … To gain better rates, that’s where scaling the # of parallel consumers comes into play, as well as ultimately clustering your RabbitMQ servers.


Coderwall

Coderwall

Different languages or different dialects?

Different languages or different dialects?

A triple dose of equality

A triple dose of equality

The whole world's gone associative on me!

Supposed you have an indexed array in PHP…
$foo = array(‘apple’, ‘orange’, ‘banana’);
And you run…
foreach ($foo as $index => $bar) {
if ($index == 1) unset($foo[$index]);
}
then the resulting array will have these indexes!:
0, 2
If you then try to iterate the array in the standard fashion:
for($i = 0; $i < count($foo); $i++) {
}
you’ll get an error when $i is 1.
The fix for resetting indexes if you ever unset in a loop is to call:
$foo = array_values($foo);
after the loop!
Lesson: EVERY array in PHP is an associative array, even “indexed” arrays.