facebook.com: Scalable memory allocation using jemalloc von Jason Evans, Montag, 3. Januar 2011 um 23:20

facebook.com: Scalable memory allocation using jemalloc von Jason Evans, Montag, 3. Januar 2011 um 23:20

Facebook Engineering wrote a note titled Scalable memory allocation using jemalloc. Read the full text here.

The Facebook website comprises a diverse set of server applications, most of which run on dedicated machines with 8+ CPU cores and 8+ GiB of RAM. These applications typically use POSIX threads for concurrent computations, with the goal of maximizing throughput by fully utilizing the CPUs and RAM. This environment poses some serious challenges for memory allocation, in particular:
Allocation and deallocation must be fast. Ideally, little memory allocation would be required in an application’s steady state, but this is far from reality for large dynamic data structures based on dynamic input. Even modest allocator improvements can have a major impact on throughput.

The relation between active memory and RAM usage must be consistent. In other words, practical bounds on allocator-induced fragmentation are critical. Consider that if fragmentation causes RAM usage to increase by 1 GiB per day, an application that is designed to leave only a little head room will fail within days.

Memory heap profiling is a critical operational aid. If all goes according to plan, leak detection and removal is a development task. But even then, dynamic input can cause unexpected memory usage spikes that can only be characterized by analyzing behavior under production loads.

In 2009, existing memory allocators met at most two of these three requirements, so we added heap profiling to jemalloc and made many optimizations, such that jemalloc is now strong on all three counts. The remainder of this post surveys jemalloc’s core algorithms and data structures before detailing numerous Facebook-motivated enhancements, followed by a real-world benchmark that compares six memory allocators under a production Web server load.

Pidgin für Jabber-Dateitransfer einrichten

Pidgin für Jabber-Dateitransfer einrichten
Als Erstes muss man sich um einen Stun-Server kümmern, damit Pidgin die öffentliche IP herausfinden kann. Dazu öffnet man den Einstellungsdialog über »Werkzeuge > Einstellungen« und wählt den Tab »Netzwerk« aus. Hier gibt man einen Stun-Server wie z.B. »stunserver.org« ein und lässt Pidgin die IP-Adresse automatisch bestimmen. Nun muss man noch eine Portweiterleitung aktivieren. Dazu gibt es zwei Wege, nämlich einmal den automatischen über UPnP (Universal Plug & Play) und einmal den manuellen. Allerdings sagt mir der UPnP-Weg wegen der mangelnden Sicherheit nicht zu. Wer dies dennoch möchte, muss UPnP bei seinem Router aktivieren und die Checkbox markieren. Beim manuellen Weiterleiten stellt man einen festen Portbereich in Pidgin ein (»Port-Bereich, auf dem gelauscht werden soll, manuell bestimmen«). Dann öffnet man die Konfiguration des Routers und leitet die Ports an den Rechner weiter (»Port-Forwarding«). Auch bei mehreren Rechnern im Netzwerk funktioniert es auch. Man stellt bei den verschiedenen Rechner verschiedene Port-Bereiche ein und leitet sie seperat an die Rechner weiter.

Aleks Totić

Aleks Totić
Back to election work. OSDV has a good shot at developing open source election machines. I am helping with design and coding. August 2006-Apr 2007: Election Designer for FairVote My unhappiness over the political system led me to focus on redistricting. Surely, I can write a program that redistricts California better. Turns out, I could not, because of there is no objective criteria to optimize for. One of the ways to really fix gerrymandering is super-districts. That’s how I got involved with Fairvote, writing an Election Designer, my first Web 2.0 app. Lessons learned: STV voting algorithms are complex. Javascript with Prototype is a lot more fun. I like Java like I used to like C, it gets well-defined job done. PHP is a mess, but a working mess, and every server framework that I’ve used is also a mess. I hate having a hashtable being my only data structure. CSS still inspires intense hate.

Dead Link: Sellaband

Sellaband
1. Discover music Browse through the site, look at the Charts, use the Search or listen to some songs and start discovering new music. 2. Support your favorite artist On SellaBand you can support your favorite artists by buying a part and helping them to raise the funds for a new music project (a new album, tour or the promotion of their music).

Aurgasm | your favorite music you’ve never heard

Aurgasm | your favorite music you’ve never heard
Aurgasm seeks to bring you an eclectic menagerie of aural pleasures. We scout out music you’ve never heard and deliver only the finest. Expect music curiously different, yet simply enjoyable. Launched in 2004, Aurgasm has developed into a destination of passionate music lovers around the globe. Its audience includes music mavens, critics, journalists, and bloggers””a highly influential audience. It was profiled in The Boston Globe and mentioned on MSNBC.com, BoingBoing, Kottke, del.icio.us, Read/Write Web, Techcrunch, Signal vs Noise, and Digg. Aurgasm specializes in everything not under the American rock umbrella—music such as: downtempo, folk, nu jazz, chanson, scandinavian, jazz, cuban, brazilian, electro, soul, jump blues, bluegrass, film score, and electronica.