Node.js w/1M concurrent connections!

I’ve decided to ramp up the Node.js experiments, and pass the 1 million concurrent connections milestone. It worked, using a swarm of 500 Amazon EC2 test clients, each establishing ~2000 active long-poll COMET connections to a single 15GB rackspace cloud server.

This isn’t landing the mars rover, or curing cancer. It’s just a pretty cool milestone IMO, which may help a few people who want to use Node.js for a large number of concurrent connections. So, hopefully it’s of some benefit to a few Node developers who can use these settings as a starting point in their own projects.

Here’s the connection count as displayed on the sprite’s page:

Here’s a sysctl dumping the number of open file handles (sockets are file handles):

Here’s the view of “top” showing system resources in use:

I think it’s pretty reasonable for 1M connections to consume 16GB of memory, but it could probably be trimmed down quite a bit. I haven’t spent any time optimizing that. I’ll leave that for another day.

Here’s a latency test run against the comet URL:

The new tweaks, placed in /etc/sysctl.conf (CentOS) and then reloaded with “sysctl -p” :

net.core.rmem_max = 33554432
net.core.wmem_max = 33554432
net.ipv4.tcp_rmem = 4096 16384 33554432
net.ipv4.tcp_wmem = 4096 16384 33554432
net.ipv4.tcp_mem = 786432 1048576 26777216
net.ipv4.tcp_max_tw_buckets = 360000
net.core.netdev_max_backlog = 2500
vm.min_free_kbytes = 65536
vm.swappiness = 0
net.ipv4.ip_local_port_range = 1024 65535

Other than that, the steps were identical to the steps described in my previous blog posts, except this time using Node.js version 0.8.3.

Here is the server source code, so you can get a sense of the complexity level. Each connected client is actively sending messages, for the purpose of verifying the connections are alive. I haven’t pushed that throughput yet, to see what data rate can be sent. Since the modest 16GB memory was consumed, this would have likely caused swapping and meant little. I’ll give it a shot with a higher memory server next time.

Escape the 1.4GB V8 heap limit in Node.jsSprites Project

autoblitz + source code

Since some folks have expressed interest in trying this bot for themselves, I decided to share the source code and binary 🙂

It’s not perfect, but you can tweak the source code (for example, changing how long rp_thread::sleep() is called, and where), and probably get some better performance. If you’re up for a challenge, you can also try to improve the code that decides on the next move! (if you do, it’d be great if you shared the changes!).

Anyway, here is the zip file. [ ]

The binaries are in the export directory. You will be asked for a “hWnd” (window handle), which can be found using the program “Spyxx” which comes with visual studio.

Game playing AI bot

Every once in a while a silly idea grips you and you decide “Screw it, I’m going to do it”.

That happened with me with a game called Bejeweled Blitz on Facebook. My friends online have been competing for high score for the past few weeks. I got to thinking…”How hard would it be to write a program to automate playing this game..?”.

This is the result of that question. It took a few days at a few hours per day. Probably about 6 hours combined time. Right now the bot is able to play pretty well. It can do better than I can most the time (although my high score currently beats the bot’s high score). With some improvements, I expect that will dramatically change.

Anyway, here’s a video of the program in action.


The autoblitz bot just scored 258,400.

Cxbx Progress

There was some amount of activity in a private branch in the Cxbx project recently..

I have been talking to a developer, Martin, who has spent some of his extra time fiddling with Cxbx. He was able to get some teaser screenshots from Battlestar Galactica. The game displays the menu, and even some in-game. I won’t get into too many details, but here are a couple screenshots.

Battlestar Galactica - menu 2 Battlestar Galactica - menu 3

Battlestar Galactica - in-game (fullscreen) Battlestar Galactica - in-game (windowed)

I have not started to work on Cxbx on a regular basis — but I think this progress is very motivating, and it is tempting me to boot back up my development setup and take another look after a very long absence.

Many thanks to Martin for the contributions and for allowing me to post about his progress. Cheers.

Chumby Antics

I have been working at Chumby for a few months now. If you have never heard about Chumby – basically, it is a small squishy wifi-internet connected device with a touchscreen and some other nifty hardware features. So, I am around Chumby devices pretty often, obviously, since it is my full time job.

Sometime’s I goof off a little..

This is a video of a small application I made on the Chumby. You touch the screen, and a small blob of “lava” follows your finger around. You basically get the sense of smearing the blobs around the touch screen.

And this is a funny video of a Chumby with two USB dogs humping it.

Function Hijacking via Export Directory

I have been working on a contract that has necessitated the use of function hooking. Basically, I need to intercept an arbitrary program’s usage of a system dll library in order to interject my own logic, and interact with the objects produced by that binary.

There is a nice tool created by Microsoft Research, called Detours. This is basically an API which helps you to perform binary function interception and instrumentation. This API is fairly robust and well thought out, and I use it in this project. However, there is certainly a fair amount of missing functionality.

While testing my application against a popular product that uses the “unicows” library, I stumbled across a very interesting situation during which the Detours method of function interception will not apply.

Basically, unicows has it’s own custom version of GetProcAddress build in. This custom code crawls through the in memory PE header and obtains function offsets by hand. This means that, for dynamically loaded function addresses, using the Detours functionality I am unable to intercept functions loaded at run-time.

So, in order to properly intercept these functions, it was necessary to create an additional API from within Detours. This function needs to crawl through the PE header, and replace the Export Directory entry for a given API with the virtual address of the function you wish to be called, instead. The function will also return the original virtual address, so that you can call that function within your intercepted version.

The new code is here: DetourReplaceExport.txt

So, now I have a working solution for hijacking an API which is linked dynamically using a non-standard GetProcAddress. Yays!