{"id":834,"date":"2008-06-04T21:15:01","date_gmt":"2008-06-05T02:15:01","guid":{"rendered":"http:\/\/www.bytebot.net\/blog\/?p=834"},"modified":"2008-06-04T21:18:11","modified_gmt":"2008-06-05T02:18:11","slug":"horizontal-scaling-with-hivedb","status":"publish","type":"post","link":"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/04\/horizontal-scaling-with-hivedb","title":{"rendered":"Horizontal Scaling with HiveDB"},"content":{"rendered":"<p>At the <a href=\"http:\/\/www.mysqlconf.com\/\">MySQL Conference &amp; Expo 2008<\/a>, Britt Crawford and Justin McCarthy, both from <a href=\"http:\/\/cafepress.com\/\">Cafepress.com<\/a>, gave us a very interesting talk on scaling with HiveDB. I took a few notes (pasted below), their <a href=\"http:\/\/assets.en.oreilly.com\/1\/event\/2\/Horizontal%20Scaling%20with%20HiveDB%20Presentation.pdf\">slides are online<\/a> (warning: 6.1MB PDF), and if you&#8217;re after their <a href=\"http:\/\/en.oreilly.com\/mysql2008\/public\/schedule\/detail\/728\">abstract<\/a> its available as well.<\/p>\n<p style=\"text-align: left;\">I also took a video of them (refer to Slide 12, for the IRC conversation):<\/p>\n<p style=\"text-align: center;\">\n<object width=\"425\" height=\"344\"><param name=\"movie\" value=\"http:\/\/www.youtube.com\/v\/8wpD3egsSOM&#038;hl=en\"><\/param><param name=\"wmode\" value=\"transparent\"><\/param><\/object>\n<\/p>\n<p>The quick notes:<\/p>\n<ul>\n<li>OLTP optimised (as it serves cafepress.com)<\/li>\n<li>Cannot lock tables, or take it offline<\/li>\n<li>Constant response time is more important than low latency (little slower query is ok, just not exponentially slower)<\/li>\n<li>Queries run might return wildly sized result sets.<\/li>\n<li>There can be growth and usage hotspots. You cannot predict this at all.<\/li>\n<li>Partition by key (the set of all partition keys is the partition dimension)<\/li>\n<li>Partitioned Hibernate from Google (Hibernate Shards). HiveDB is now married up with shards.<\/li>\n<li>Thought about MySQL Proxy to support high availability components, but it was dismissed<\/li>\n<\/ul>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li class=\"share-email\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-email sd-button share-icon\" href=\"mailto:?subject=%5BShared%20Post%5D%20Horizontal%20Scaling%20with%20HiveDB&body=https%3A%2F%2Fwww.bytebot.net%2Fblog%2Farchives%2F2008%2F06%2F04%2Fhorizontal-scaling-with-hivedb&share=email\" target=\"_blank\" title=\"Click to email a link to a friend\" data-email-share-error-title=\"Do you have email set up?\" data-email-share-error-text=\"If you&#039;re having problems sharing via email, you might not have email set up for your browser. You may need to create a new email yourself.\" data-email-share-nonce=\"a9db23e06b\" data-email-share-track-url=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/04\/horizontal-scaling-with-hivedb?share=email\"><span>Email<\/span><\/a><\/li><li class=\"share-facebook\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-facebook-834\" class=\"share-facebook sd-button share-icon\" href=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/04\/horizontal-scaling-with-hivedb?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\" ><span>Facebook<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-linkedin-834\" class=\"share-linkedin sd-button share-icon\" href=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/04\/horizontal-scaling-with-hivedb?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\" ><span>LinkedIn<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-twitter-834\" class=\"share-twitter sd-button share-icon\" href=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/04\/horizontal-scaling-with-hivedb?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\" ><span>Twitter<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>At the MySQL Conference &amp; Expo 2008, Britt Crawford and Justin McCarthy, both from Cafepress.com, gave us a very interesting talk on scaling with HiveDB. I took a few notes (pasted below), their slides are online (warning: 6.1MB PDF), and if you&#8217;re after their abstract its available as well. I also took a video of [&hellip;]<\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li class=\"share-email\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-email sd-button share-icon\" href=\"mailto:?subject=%5BShared%20Post%5D%20Horizontal%20Scaling%20with%20HiveDB&body=https%3A%2F%2Fwww.bytebot.net%2Fblog%2Farchives%2F2008%2F06%2F04%2Fhorizontal-scaling-with-hivedb&share=email\" target=\"_blank\" title=\"Click to email a link to a friend\" data-email-share-error-title=\"Do you have email set up?\" data-email-share-error-text=\"If you&#039;re having problems sharing via email, you might not have email set up for your browser. You may need to create a new email yourself.\" data-email-share-nonce=\"a9db23e06b\" data-email-share-track-url=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/04\/horizontal-scaling-with-hivedb?share=email\"><span>Email<\/span><\/a><\/li><li class=\"share-facebook\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-facebook-834\" class=\"share-facebook sd-button share-icon\" href=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/04\/horizontal-scaling-with-hivedb?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\" ><span>Facebook<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-linkedin-834\" class=\"share-linkedin sd-button share-icon\" href=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/04\/horizontal-scaling-with-hivedb?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\" ><span>LinkedIn<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-twitter-834\" class=\"share-twitter sd-button share-icon\" href=\"https:\/\/www.bytebot.net\/blog\/archives\/2008\/06\/04\/horizontal-scaling-with-hivedb?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\" ><span>Twitter<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_options":[]},"categories":[23],"tags":[221,220,56,224,225,226,223,227,219,222,212],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p4vJD-ds","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":544,"url":"https:\/\/www.bytebot.net\/blog\/archives\/2007\/05\/02\/scaling-mysql-presentations","url_meta":{"origin":834,"position":0},"title":"Scaling MySQL presentations","date":"2\/5\/2007","format":false,"excerpt":"Everyone likes to scale - Peter Van Dijck has got some top 10 presentations listed - Twitter, Flickr, LiveJournal, Six Apart (Vox), Last.fm, SlideShare, etc. Guess what these sites are all generally backed by? You guessed right - go MySQL. I however didn't know that Bloglines was backed by Sleepycat.\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":538,"url":"https:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/27\/diggcom-scales-japanese-character-set-data-warehousing","url_meta":{"origin":834,"position":1},"title":"Digg.com scales; Japanese Character Set; Data Warehousing","date":"27\/4\/2007","format":false,"excerpt":"I missed a couple of talks that I'd really have liked to attend, for various reasons (probably the fact that at the MySQL conferences, staff also have a tonne of meetings and customers\/people to meet). Thanks to the great bloggers, I don't feel so bad for missing such talks. And\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":434,"url":"https:\/\/www.bytebot.net\/blog\/archives\/2006\/10\/18\/free-ride-to-mysql-camp","url_meta":{"origin":834,"position":2},"title":"Free ride to MySQL Camp","date":"18\/10\/2006","format":false,"excerpt":"I don't normally blog about events that I'm not going to be attending, but this seemed like too cool an offer to pass up. Proven Scaling will be offering one free ticket to MySQL Camp - thats free airfare and hotel, for the rather cool MySQL Camp un-conference. From what\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":607,"url":"https:\/\/www.bytebot.net\/blog\/archives\/2007\/08\/08\/high-scalability","url_meta":{"origin":834,"position":3},"title":"High Scalability","date":"8\/8\/2007","format":false,"excerpt":"I think anyone into databases, Linux and so on (i.e. the LAMP stack), will benefit from the High Scalability website. Heck, any web developer will. Its well tagged (say, you're into mysql or memcached), has a lot of content, and links to other sites that it gets its summaries\/information from.\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":539,"url":"https:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/30\/capacity-planning-for-lamp-or-more-flickr-innards","url_meta":{"origin":834,"position":4},"title":"Capacity Planning For LAMP or more Flickr-innards","date":"30\/4\/2007","format":false,"excerpt":"From the \"I wish I attended this talk\" department. When I attended Dathan's talk about Federation at Flickr, he did mention that the next day there was going to be a really interesting talk by John Allspaw, who is an Engineering Manager at Flickr. John has posted up his slides,\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1357,"url":"https:\/\/www.bytebot.net\/blog\/archives\/2009\/03\/01\/posterous-and-friendfeed-talk-infrastructure","url_meta":{"origin":834,"position":5},"title":"Posterous and FriendFeed talk infrastructure","date":"1\/3\/2009","format":false,"excerpt":"A couple interesting things coming out of startup land. For one, Posterous has a little writeup on Building and Scaling a Startup on Rails: 12 Things We Learned the Hard Way. Good things to take away include using Sphinx\/Solr for search, but the real important takeaway for the MySQL crowd\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/posts\/834"}],"collection":[{"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/comments?post=834"}],"version-history":[{"count":0,"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/posts\/834\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/media?parent=834"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/categories?post=834"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/tags?post=834"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}