{"id":847,"date":"2008-06-25T22:52:46","date_gmt":"2008-06-26T03:52:46","guid":{"rendered":"http:\/\/www.bytebot.net\/blog\/?p=847"},"modified":"2008-06-25T22:52:46","modified_gmt":"2008-06-26T03:52:46","slug":"how-facebook-serves-pictures","status":"publish","type":"post","link":"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/25\/how-facebook-serves-pictures","title":{"rendered":"How Facebook serves pictures"},"content":{"rendered":"<p>I caught <a href=\"http:\/\/static.flowgram.com\/p2.html#2qi3k8eicrfgkv\">Facebook &#8211; Needle in a Haystack: Efficient Storage of Billions of Photos<\/a> on Flowgram. First up, I&#8217;m not a big fan of Flowgrams &#8211; the format is sensible, slide and voice, is excellent, but the delivery in a web browser isn&#8217;t optimal&#8230; make downloadable videos!<\/p>\n<p>The talk however, was excellent. Do watch it, and learn a bit more about Facebook&#8217;s infrastructure. Anyway, some notes I took from the talk:<\/p>\n<ul>\n<li>&#8220;We&#8217;re one of the largest MySQL installations in the world&#8221;<\/li>\n<li>Use memcache &#8211; &#8220;We have memcache because databases aren&#8217;t fast&#8221; (later on in the questions)<\/li>\n<li>Separate team focusing on APE (Apache, PHP and Extensions that they work on)<\/li>\n<li>6.5 billion total images, 4-5 sizes stored for each, so 30 billion files, of about 540TB total&#8230; During peak? 475,000 images served per second, and growing by 100 million uploads per week<\/li>\n<li>Images are usually pulled from a Content Delivery Network (CDN), so it reduces the request rate on their servers<\/li>\n<li>They use NetApp Storage, but basically their upload servers speak NFS to write to NetApp.<\/li>\n<li>Cachr (evhttp based) and File Handle Cache use memcache as a backing store&#8230; FHC is based on lighttpd!<\/li>\n<li>Makes use of a &#8220;haystack&#8221; &#8211; user-level abstraction, storing a separate index file that has more efficient metadata (to reduce disk seeks &#8211; 1 disk seek or less for any workload). Pretty deep in the discussion of the haystack server architecture, also evhttp-based<\/li>\n<li>MySQL use? Very few transactions, very few joins<\/li>\n<li>Video is a very different beast, and the design is a little different<\/li>\n<\/ul>\n<p>If you&#8217;re into information about photo storage sites, don&#8217;t hesitate to also read my previous <a href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/25\/federation-at-flickr-a-tour-of-the-flickr-architecture\">notes on Flickr<\/a>.<\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li class=\"share-email\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-email sd-button share-icon\" href=\"mailto:?subject=%5BShared%20Post%5D%20How%20Facebook%20serves%20pictures&body=https%3A%2F%2Fwww.bytebot.net%2Fblog%2Farchives%2F2008%2F06%2F25%2Fhow-facebook-serves-pictures&share=email\" target=\"_blank\" title=\"Click to email a link to a friend\" data-email-share-error-title=\"Do you have email set up?\" data-email-share-error-text=\"If you&#039;re having problems sharing via email, you might not have email set up for your browser. You may need to create a new email yourself.\" data-email-share-nonce=\"e7d01b3ea6\" data-email-share-track-url=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/25\/how-facebook-serves-pictures?share=email\"><span>Email<\/span><\/a><\/li><li class=\"share-facebook\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-facebook-847\" class=\"share-facebook sd-button share-icon\" href=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/25\/how-facebook-serves-pictures?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\" ><span>Facebook<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-linkedin-847\" class=\"share-linkedin sd-button share-icon\" href=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/25\/how-facebook-serves-pictures?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\" ><span>LinkedIn<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-twitter-847\" class=\"share-twitter sd-button share-icon\" href=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/25\/how-facebook-serves-pictures?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\" ><span>Twitter<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>I caught Facebook &#8211; Needle in a Haystack: Efficient Storage of Billions of Photos on Flowgram. First up, I&#8217;m not a big fan of Flowgrams &#8211; the format is sensible, slide and voice, is excellent, but the delivery in a web browser isn&#8217;t optimal&#8230; make downloadable videos! The talk however, was excellent. Do watch it, [&hellip;]<\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li class=\"share-email\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-email sd-button share-icon\" href=\"mailto:?subject=%5BShared%20Post%5D%20How%20Facebook%20serves%20pictures&body=https%3A%2F%2Fwww.bytebot.net%2Fblog%2Farchives%2F2008%2F06%2F25%2Fhow-facebook-serves-pictures&share=email\" target=\"_blank\" title=\"Click to email a link to a friend\" data-email-share-error-title=\"Do you have email set up?\" data-email-share-error-text=\"If you&#039;re having problems sharing via email, you might not have email set up for your browser. You may need to create a new email yourself.\" data-email-share-nonce=\"e7d01b3ea6\" data-email-share-track-url=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/25\/how-facebook-serves-pictures?share=email\"><span>Email<\/span><\/a><\/li><li class=\"share-facebook\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-facebook-847\" class=\"share-facebook sd-button share-icon\" href=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/25\/how-facebook-serves-pictures?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\" ><span>Facebook<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-linkedin-847\" class=\"share-linkedin sd-button share-icon\" href=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/25\/how-facebook-serves-pictures?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\" ><span>LinkedIn<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-twitter-847\" class=\"share-twitter sd-button share-icon\" href=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/25\/how-facebook-serves-pictures?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\" ><span>Twitter<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_options":[]},"categories":[23],"tags":[300,297,301,299,298,1775],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p4vJD-dF","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":707,"url":"https:\/\/www.bytebot.net\/blog\/archives\/2008\/01\/28\/memcache-keeping-data-in-the-handiest-place-memory","url_meta":{"origin":847,"position":0},"title":"Memcache, keeping data in the handiest place: memory","date":"28\/1\/2008","format":false,"excerpt":"While I ducked out of Giuseppe's miniconf talk, on MySQL Proxy (a great session, might I add - it takes up 2 slots right up until lunch), I went over to the LinuxChix miniconf, to attend a talk about memcache, by Brenda Wallace. Brenda, works at Catalyst IT, in New\u2026","rel":"","context":"In &quot;General&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":716,"url":"https:\/\/www.bytebot.net\/blog\/archives\/2008\/02\/14\/mysql-miniconf-videos-from-linuxconfau-2008","url_meta":{"origin":847,"position":1},"title":"MySQL Miniconf videos, from linux.conf.au 2008","date":"14\/2\/2008","format":false,"excerpt":"If you couldn't make the MySQL MiniConf at linux.conf.au 2008, no worries - there's recorded video!Upcoming MySQL Features (Stewart Smith) - focused on the future, as opposed to what's available right now\/soonMySQL Indexing Methods (Jonathon Coombes) AND MySQL Optimisation by Design (Arjen Lentz)- the slides are viewable from the video,\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":528,"url":"https:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/23\/scaling-twitter-is-twitter-is-udp-or-tcp-its-definitely-udp","url_meta":{"origin":847,"position":2},"title":"Scaling Twitter: &#8220;Is Twitter is UDP or TCP? Its definitely UDP.&#8221;","date":"23\/4\/2007","format":false,"excerpt":"Presented by Blaine Cook, a developer from Odeo, now probably CTO of Twitter (Obvious Corp spawed, I think). There's a video and slides (yes, you need evil Flash so I haven't viewed it myself). Then there are my notes... possibly with some thoughts attached to them. No, they're not organized,\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1432,"url":"https:\/\/www.bytebot.net\/blog\/archives\/2009\/04\/17\/frank-mashraqi-on-hadoop-memcached-and-why-the-mysql-conference-is-cool","url_meta":{"origin":847,"position":3},"title":"Frank Mashraqi on Hadoop, memcached, and why the MySQL Conference is cool","date":"17\/4\/2009","format":false,"excerpt":"Today I spoke with Farhan \"Frank\" Mashraqi, former Fotolog DBA, now working at a startup, NetEdge, working on social analytics. He's talking about the two sessions he's giving next week at the MySQL Conference & Expo 2009, as well as the benefits of being at the MySQL Conference & Expo.\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":536,"url":"https:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/26\/extreme-makeover-database-or-mysqlyoutube","url_meta":{"origin":847,"position":4},"title":"Extreme Makeover: Database or MySQL@YouTube","date":"26\/4\/2007","format":false,"excerpt":"Arguably one of the most interesting keynotes (and technical to boot!), Paul Tuckfield not only entertained us in his 40 minute keynote, he also did so outside when the keynotes ended. Just the DBA at PayPal, just the DBA at YouTube. Only 3 DBAs at YouTube that make it all\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1018,"url":"https:\/\/www.bytebot.net\/blog\/archives\/2008\/09\/11\/project-kenai","url_meta":{"origin":847,"position":5},"title":"Project Kenai","date":"11\/9\/2008","format":false,"excerpt":"Sun is a huge company. So it comes as no surprise that I'm finding out about Project Kenai via Tim Bray, instead of some internal mailing list (believe me, there must be thousands). Tim's got a Q&A with Nick Sieger, who's one of the chieftains behind Kenai. I find it\u2026","rel":"","context":"In &quot;General&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/posts\/847"}],"collection":[{"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/comments?post=847"}],"version-history":[{"count":0,"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/posts\/847\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/media?parent=847"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/categories?post=847"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/tags?post=847"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}