{"id":539,"date":"2007-04-30T22:27:01","date_gmt":"2007-05-01T03:27:01","guid":{"rendered":"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/30\/capacity-planning-for-lamp-or-more-flickr-innards"},"modified":"2007-04-30T22:27:06","modified_gmt":"2007-05-01T03:27:06","slug":"capacity-planning-for-lamp-or-more-flickr-innards","status":"publish","type":"post","link":"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/30\/capacity-planning-for-lamp-or-more-flickr-innards","title":{"rendered":"Capacity Planning For LAMP or more Flickr-innards"},"content":{"rendered":"<p>From the &#8220;I wish I attended this talk&#8221; department. When I attended Dathan&#8217;s talk about <a href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/25\/federation-at-flickr-a-tour-of-the-flickr-architecture\">Federation at Flickr<\/a>, he did mention that the next day there was going to be a really interesting talk by John Allspaw, who is an Engineering Manager at Flickr. John has posted up his <a href=\"http:\/\/www.kitchensoap.com\/2007\/04\/27\/slides-from-capacity-planning-for-lamp-talk-at-mysql-conf-2007\/\">slides<\/a>, which I might add, are quite an interesting read (look at the speaker notes, they sort of give a heads up as to what you missed).<\/p>\n<p>Its a pity no one made notes of this on Planet MySQL, so if anyone did attend the talk and you did take notes, please do place them online! This talk isn&#8217;t so much teaching you about capacity planning, its really more about all the pointers you can take away, about monitoring, graphing statistics, the fun of deployment, and of course some fun Flickr statistics (on slide 6, 18).<\/p>\n<p>Pictures are fun! Capacity is not the same as speed (and it doesn&#8217;t mean performance). He mentions that you probably don&#8217;t want to read up about queuing theory (and should probably forget about benchmarks), because its mostly irrelevant to the real world. Testing in production is good, so don&#8217;t be afraid (I guess this is why Flickr has built-in notifications now).<\/p>\n<p>Tools of interest: <a href=\"http:\/\/ganglia.sourceforge.net\/\">Ganglia<\/a>, for pretty graphs, rrdtool, memcached, <a href=\"http:\/\/www.arizona-software.ch\/applications\/graphclick\/en\/\">GraphClick<\/a> (OS X only, and I wonder why they only get MRTG information in graphs, with no raw data at Yahoo!?). For deployment they use SystemImager (oh, I remember using this &#8211; it gets &#8220;fun&#8221; when you try to image dual-boot machines, but thats another story), and <a href=\"http:\/\/code.google.com\/p\/subcon\/\">Subcon<\/a> (interesting, I wonder how this compares to Slack).<\/p>\n<p>Technorati Tags: <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/mysqlconf\" rel=\"tag\">mysqlconf<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/mysqlconf07\" rel=\"tag\">mysqlconf07<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/flickr\" rel=\"tag\">flickr<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/capacity\" rel=\"tag\">capacity<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/planning\" rel=\"tag\">planning<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/scaling\" rel=\"tag\">scaling<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/LAMP\" rel=\"tag\">LAMP<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/mysql\" rel=\"tag\">mysql<\/a><\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li class=\"share-email\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-email sd-button share-icon\" href=\"mailto:?subject=%5BShared%20Post%5D%20Capacity%20Planning%20For%20LAMP%20or%20more%20Flickr-innards&body=http%3A%2F%2Fwww.bytebot.net%2Fblog%2Farchives%2F2007%2F04%2F30%2Fcapacity-planning-for-lamp-or-more-flickr-innards&share=email\" target=\"_blank\" title=\"Click to email a link to a friend\" data-email-share-error-title=\"Do you have email set up?\" data-email-share-error-text=\"If you&#039;re having problems sharing via email, you might not have email set up for your browser. You may need to create a new email yourself.\" data-email-share-nonce=\"49a7366af8\" data-email-share-track-url=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/30\/capacity-planning-for-lamp-or-more-flickr-innards?share=email\"><span>Email<\/span><\/a><\/li><li class=\"share-facebook\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-facebook-539\" class=\"share-facebook sd-button share-icon\" href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/30\/capacity-planning-for-lamp-or-more-flickr-innards?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\" ><span>Facebook<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-linkedin-539\" class=\"share-linkedin sd-button share-icon\" href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/30\/capacity-planning-for-lamp-or-more-flickr-innards?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\" ><span>LinkedIn<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-twitter-539\" class=\"share-twitter sd-button share-icon\" href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/30\/capacity-planning-for-lamp-or-more-flickr-innards?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\" ><span>Twitter<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>From the &#8220;I wish I attended this talk&#8221; department. When I attended Dathan&#8217;s talk about Federation at Flickr, he did mention that the next day there was going to be a really interesting talk by John Allspaw, who is an Engineering Manager at Flickr. John has posted up his slides, which I might add, are [&hellip;]<\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li class=\"share-email\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-email sd-button share-icon\" href=\"mailto:?subject=%5BShared%20Post%5D%20Capacity%20Planning%20For%20LAMP%20or%20more%20Flickr-innards&body=http%3A%2F%2Fwww.bytebot.net%2Fblog%2Farchives%2F2007%2F04%2F30%2Fcapacity-planning-for-lamp-or-more-flickr-innards&share=email\" target=\"_blank\" title=\"Click to email a link to a friend\" data-email-share-error-title=\"Do you have email set up?\" data-email-share-error-text=\"If you&#039;re having problems sharing via email, you might not have email set up for your browser. You may need to create a new email yourself.\" data-email-share-nonce=\"49a7366af8\" data-email-share-track-url=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/30\/capacity-planning-for-lamp-or-more-flickr-innards?share=email\"><span>Email<\/span><\/a><\/li><li class=\"share-facebook\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-facebook-539\" class=\"share-facebook sd-button share-icon\" href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/30\/capacity-planning-for-lamp-or-more-flickr-innards?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\" ><span>Facebook<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-linkedin-539\" class=\"share-linkedin sd-button share-icon\" href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/30\/capacity-planning-for-lamp-or-more-flickr-innards?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\" ><span>LinkedIn<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-twitter-539\" class=\"share-twitter sd-button share-icon\" href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/30\/capacity-planning-for-lamp-or-more-flickr-innards?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\" ><span>Twitter<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_options":[]},"categories":[23],"tags":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p4vJD-8H","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":1271,"url":"http:\/\/www.bytebot.net\/blog\/archives\/2009\/01\/13\/some-tabs-marten-interview-facebook-flickr","url_meta":{"origin":539,"position":0},"title":"Some tabs &#8211; Marten interview, Facebook, Flickr","date":"13\/1\/2009","format":false,"excerpt":"I've been collecting a bunch of tabs, MySQL related, that I think people might have missed during the holiday period. Contrarian Minds: Marten Mickos - this is a great interview with former CEO of MySQL, now SVP of the Database Group, at Sun Microsystems. Its got a bit of interesting\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":533,"url":"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/25\/federation-at-flickr-a-tour-of-the-flickr-architecture","url_meta":{"origin":539,"position":1},"title":"Federation at Flickr: A tour of the Flickr Architecture","date":"25\/4\/2007","format":false,"excerpt":"I've always been a big Flickr fan, and user, and love them even more now that their 2GB limit is gone. So this was a most interesting talk, and I think photographers and Flickr users alike will find it interesting what's behind Flickr. Dathan also has a very interesting blog.\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":558,"url":"http:\/\/www.bytebot.net\/blog\/archives\/2007\/05\/23\/some-random-thoughts-notes-etc-from-the-mysql-conference-expo-2007","url_meta":{"origin":539,"position":2},"title":"Some random thoughts, notes, etc. from the MySQL Conference &#038; Expo 2007","date":"23\/5\/2007","format":false,"excerpt":"Here are some random thoughts, notes, observations, etc.Jay Pipes - he made the conference possibleFriendster uses Bugzilla internally. Yes they're still alive, even though MySpace and so forth are around and kicking. Had to Google them (I wanted to find their old talk about their storage engine), and found MySQL\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":847,"url":"http:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/25\/how-facebook-serves-pictures","url_meta":{"origin":539,"position":3},"title":"How Facebook serves pictures","date":"25\/6\/2008","format":false,"excerpt":"I caught Facebook - Needle in a Haystack: Efficient Storage of Billions of Photos on Flowgram. First up, I'm not a big fan of Flowgrams - the format is sensible, slide and voice, is excellent, but the delivery in a web browser isn't optimal... make downloadable videos! The talk however,\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1720,"url":"http:\/\/www.bytebot.net\/blog\/archives\/2010\/03\/07\/abusing-mysql-thoughts-on-nosql","url_meta":{"origin":539,"position":4},"title":"Abusing MySQL (&#038; thoughts on NoSQL)","date":"7\/3\/2010","format":false,"excerpt":"The NoSQL\/relational database debate has been going on for quite some time. MariaDB, like MySQL is relational. And if you read these series of blog posts, you'll realise that if you use MySQL correctly, you can achieve quite a lot. It all starts with Kellan Elliott-McCrea with his introductory post\u2026","rel":"","context":"In &quot;Databases&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2247,"url":"http:\/\/www.bytebot.net\/blog\/archives\/2012\/02\/05\/new-mysql-5-6-features-by-oli-sennhauser","url_meta":{"origin":539,"position":5},"title":"New MySQL 5.6 Features by Oli Sennhauser","date":"5\/2\/2012","format":false,"excerpt":"First talk at FOSDEM MySQL Devroom by Oli Sennhauser, of FromDual. Quick notes\/liveblog of the talk, plus links from a quick search. New Release Model: starts with at least in beta quality, milestone releases are RC quality (every 3-6 months), between milestones new features are allowed, GA releases every 12-18\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"amp_enabled":true,"_links":{"self":[{"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/posts\/539"}],"collection":[{"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/comments?post=539"}],"version-history":[{"count":0,"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/posts\/539\/revisions"}],"wp:attachment":[{"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/media?parent=539"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/categories?post=539"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/tags?post=539"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}