{"id":550,"date":"2007-05-07T23:35:50","date_gmt":"2007-05-08T04:35:50","guid":{"rendered":"http:\/\/www.bytebot.net\/blog\/archives\/2007\/05\/07\/yahoo-pipes-the-edwin-pipe-in-under-15-minutes"},"modified":"2007-05-07T23:35:53","modified_gmt":"2007-05-08T04:35:53","slug":"yahoo-pipes-the-edwin-pipe-in-under-15-minutes","status":"publish","type":"post","link":"http:\/\/www.bytebot.net\/blog\/archives\/2007\/05\/07\/yahoo-pipes-the-edwin-pipe-in-under-15-minutes","title":{"rendered":"Yahoo! Pipes &#8211; the Edwin Pipe in under 15 minutes"},"content":{"rendered":"<p>At the MySQL Conference the closing keynote was on Yahoo! <a href=\"http:\/\/pipes.yahoo.com\/pipes\/\">Pipes<\/a>, by Pasha Sadri, a Principal Software Engineer, Advanced Development Division, Yahoo!. I wanted to try it, but I was on Firefox 1.5 on Fedora Core 6 and there was no way I was going to build a pipe during the talk. <\/p>\n<p>Fast forward a week or so later, and a boring Friday night ensued. What better thing to do, than to play with Pipes. In under fifteen minutes, I created the <a href=\"http:\/\/pipes.yahoo.com\/pipes\/pipe.info?_id=otVKHRb62xG_KO9Vy6ky6g\">Edwin Pipe<\/a>. What is it? Its a pipe that is all things MySQL &#8211; comprehensive source of news, whats cool, and so forth. There are some limitations &#8211; regular expression support is supposedly like Perl&#8217;s, but is not quite complete. The Unique operator is pretty cool, filtering is good (can be improved with better regex support), and maybe some sort of fuzzyness in the way data is displayed (I don&#8217;t only want all Digg mysql related items popping up at the top, or I don&#8217;t only want all mysql job forum details at the bottom, etc.). Language conversion via a Babelfish operator exists, but not language filtering (maybe I only want all English text displayed in my final pipe output).<\/p>\n<p>That aside, the forums are pretty active. Pipes are ridiculously easy to create. Its simply great stuff. Oh, shorter URLs &#8211; the URLs are so long and not feasible, in my opinion. Impressive is the support to then get RSS output, and also JSON (so all processing is done on the server side). Happy I am with sites that provide JSON feeds.<\/p>\n<p>Now, for some notes I took during the closing keynote.<\/p>\n<ul>\n<li>A while ago, he wanted to find an apartment near a park. Go to Craigslist and find apartment lists, then click the map link, and also check distance to a park on the map&#8230; This is tedious, and not automated.<\/li>\n<li>Craigslist apartment RSS feed. Yahoo! Local API to find Parks. Why not tie this in together? It started with about 50 lines of Perl code, and it combined feeds + web services (this is your Web 2.0 mashup).<\/li>\n<li>Pipes: free online service that lets you remix data and create mashups using a <b>visual editor<\/b>.<\/li>\n<li>Pipes treats the web as a big database, as they do joins across different &#8216;tables&#8217;.<\/li>\n<li><i><b>Design principles came from the Unix Pipes<\/b><\/i>. They&#8217;re like pipes for the Web. Build useful applications from simple primitives.<\/li>\n<li>The more open Pipes is, the more useful it will be (so Google goodness will also work).<\/li>\n<li>Output available in JSON, so it can be used as another application. Get email or SMS from output, even. RSS is obviously available.<\/li>\n<li>App Examples: Last.fm + Flickr, Babbler (Second Life, language translation) by Max Case.<\/li>\n<li>Must enable users to solve ad-hoc problems. <i><b>User generated &#8220;features&#8221; and disposable applications -&gt; the future<\/b><\/i>.<\/li>\n<li>Pipes uses MySQL, squid for caching, PHP &amp; Perl (lots of CPAN modules) for serving and back-end processing of the pipes.<\/li>\n<\/ul>\n<p>Edwin 2.0 is already in the works. It will have more cool feeds, and probably work out all the language issues with more separated regexes. More fuzzy organizing of data, if possible. If you want to see a <a href=\"http:\/\/pipes.yahoo.com\/pipes\/pipe.info?_id=DpD6nWr02xGcHjQNo_NLYQ\">MySQL Blogger Photo Gallery<\/a>, bman_seattle created a pipe too.<\/p>\n<p>Technorati Tags: <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/mysqlconf\" rel=\"tag\">mysqlconf<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/mysqlconf07\" rel=\"tag\">mysqlconf07<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/yahoo\" rel=\"tag\">yahoo<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/pipes\" rel=\"tag\">pipes<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/edwin%20pipe\" rel=\"tag\">edwin pipe<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/json\" rel=\"tag\">json<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/visual%20editing\" rel=\"tag\">visual editing<\/a>, <a class=\"performancingtags\" href=\"http:\/\/technorati.com\/tag\/mashup\" rel=\"tag\">mashup<\/a><\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li class=\"share-email\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-email sd-button share-icon\" href=\"mailto:?subject=%5BShared%20Post%5D%20Yahoo%21%20Pipes%20-%20the%20Edwin%20Pipe%20in%20under%2015%20minutes&body=http%3A%2F%2Fwww.bytebot.net%2Fblog%2Farchives%2F2007%2F05%2F07%2Fyahoo-pipes-the-edwin-pipe-in-under-15-minutes&share=email\" target=\"_blank\" title=\"Click to email a link to a friend\" data-email-share-error-title=\"Do you have email set up?\" data-email-share-error-text=\"If you&#039;re having problems sharing via email, you might not have email set up for your browser. You may need to create a new email yourself.\" data-email-share-nonce=\"a197cbd381\" data-email-share-track-url=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/05\/07\/yahoo-pipes-the-edwin-pipe-in-under-15-minutes?share=email\"><span>Email<\/span><\/a><\/li><li class=\"share-facebook\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-facebook-550\" class=\"share-facebook sd-button share-icon\" href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/05\/07\/yahoo-pipes-the-edwin-pipe-in-under-15-minutes?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\" ><span>Facebook<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-linkedin-550\" class=\"share-linkedin sd-button share-icon\" href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/05\/07\/yahoo-pipes-the-edwin-pipe-in-under-15-minutes?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\" ><span>LinkedIn<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-twitter-550\" class=\"share-twitter sd-button share-icon\" href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/05\/07\/yahoo-pipes-the-edwin-pipe-in-under-15-minutes?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\" ><span>Twitter<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>At the MySQL Conference the closing keynote was on Yahoo! Pipes, by Pasha Sadri, a Principal Software Engineer, Advanced Development Division, Yahoo!. I wanted to try it, but I was on Firefox 1.5 on Fedora Core 6 and there was no way I was going to build a pipe during the talk. Fast forward a [&hellip;]<\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li class=\"share-email\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-email sd-button share-icon\" href=\"mailto:?subject=%5BShared%20Post%5D%20Yahoo%21%20Pipes%20-%20the%20Edwin%20Pipe%20in%20under%2015%20minutes&body=http%3A%2F%2Fwww.bytebot.net%2Fblog%2Farchives%2F2007%2F05%2F07%2Fyahoo-pipes-the-edwin-pipe-in-under-15-minutes&share=email\" target=\"_blank\" title=\"Click to email a link to a friend\" data-email-share-error-title=\"Do you have email set up?\" data-email-share-error-text=\"If you&#039;re having problems sharing via email, you might not have email set up for your browser. You may need to create a new email yourself.\" data-email-share-nonce=\"a197cbd381\" data-email-share-track-url=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/05\/07\/yahoo-pipes-the-edwin-pipe-in-under-15-minutes?share=email\"><span>Email<\/span><\/a><\/li><li class=\"share-facebook\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-facebook-550\" class=\"share-facebook sd-button share-icon\" href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/05\/07\/yahoo-pipes-the-edwin-pipe-in-under-15-minutes?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\" ><span>Facebook<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-linkedin-550\" class=\"share-linkedin sd-button share-icon\" href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/05\/07\/yahoo-pipes-the-edwin-pipe-in-under-15-minutes?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\" ><span>LinkedIn<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-twitter-550\" class=\"share-twitter sd-button share-icon\" href=\"http:\/\/www.bytebot.net\/blog\/archives\/2007\/05\/07\/yahoo-pipes-the-edwin-pipe-in-under-15-minutes?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\" ><span>Twitter<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_options":[]},"categories":[23],"tags":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p4vJD-8S","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":1064,"url":"http:\/\/www.bytebot.net\/blog\/archives\/2008\/09\/23\/rails-fails-sticker","url_meta":{"origin":550,"position":0},"title":"Rails&#8230; Fails&#8230; (sticker)","date":"23\/9\/2008","format":false,"excerpt":"I had first seen the interesting Rails logo in a talk by Terry Chay, while I was at OSCON, a few months ago. Now, my esteemed colleague Jay Pipes has it on his laptop. It seems they're making stickers, even. Otherwise, my next task is to revamp our Ruby content.\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2426,"url":"http:\/\/www.bytebot.net\/blog\/archives\/2012\/07\/15\/eulogy-for-mysql-forge","url_meta":{"origin":550,"position":1},"title":"eulogy for mysql forge","date":"15\/7\/2012","format":false,"excerpt":"When the mysql librarian closed, I didn't think too much about it; it was a feature I probably never used. However this month brings the end of the mysql forge. The MySQL Forge was something I worked on while I was at MySQL so I am a little sad to\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":558,"url":"http:\/\/www.bytebot.net\/blog\/archives\/2007\/05\/23\/some-random-thoughts-notes-etc-from-the-mysql-conference-expo-2007","url_meta":{"origin":550,"position":2},"title":"Some random thoughts, notes, etc. from the MySQL Conference &#038; Expo 2007","date":"23\/5\/2007","format":false,"excerpt":"Here are some random thoughts, notes, observations, etc.Jay Pipes - he made the conference possibleFriendster uses Bugzilla internally. Yes they're still alive, even though MySpace and so forth are around and kicking. Had to Google them (I wanted to find their old talk about their storage engine), and found MySQL\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":524,"url":"http:\/\/www.bytebot.net\/blog\/archives\/2007\/04\/21\/business-scripting-languages-or-saps-marketing-talk-stanford-hci-mashup","url_meta":{"origin":550,"position":3},"title":"Business Scripting Languages or SAP&#8217;s marketing talk + Stanford HCI mashup","date":"21\/4\/2007","format":false,"excerpt":"The after lunch talk on Business Scripting Languages, by Asuman Suenbuel and Murray Spork was something I found very hard to stay awake in. In fact, I think so did many others, some of whom walked out of the room.The first half of the talk was filled with SAP marketing\u2026","rel":"","context":"In &quot;General&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1490,"url":"http:\/\/www.bytebot.net\/blog\/archives\/2009\/05\/07\/interview-with-rohit-nadhani-founder-of-webyog","url_meta":{"origin":550,"position":4},"title":"Interview with Rohit Nadhani, founder of Webyog","date":"7\/5\/2009","format":false,"excerpt":"At the MySQL Conference, I had the chance to interview Rohit Nadhani, founder of Webyog, the folk that make SQLyog and MONyog. Watch the video, for more. Generally based out of Bangalore (I visited them when I was last there), Webyog just started an office in Santa Clara, and are\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":761,"url":"http:\/\/www.bytebot.net\/blog\/archives\/2008\/03\/31\/hitchhackers-guide-to-the-mysql-conference-rubyruby-on-rails-edition","url_meta":{"origin":550,"position":5},"title":"Hitchhacker&#8217;s Guide to the MySQL Conference &#8211; Ruby\/Ruby on Rails Edition","date":"31\/3\/2008","format":false,"excerpt":"The Hitchhacker's Guide to the MySQL Conference Ruby\/Ruby on Rails Developer's Edition Following on from the excellent Hitchhacker's Guide to the MySQL Conference - PHP Edition by Jay Pipes, I'm doing one for the Ruby and Ruby on Rails developer in you. If you haven't registered yet, remember, I can\u2026","rel":"","context":"In &quot;MySQL&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"amp_enabled":true,"_links":{"self":[{"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/posts\/550"}],"collection":[{"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/comments?post=550"}],"version-history":[{"count":0,"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/posts\/550\/revisions"}],"wp:attachment":[{"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/media?parent=550"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/categories?post=550"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.bytebot.net\/blog\/wp-json\/wp\/v2\/tags?post=550"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}