Screen-Scraping With CSS Selectors in PHP

Jake Marsh — Wed, 08 Jul 2009 01:36:55 +0000

When it comes to programming techniques, screen-scraping can be a complicated and annoying thing to deal with. I've dealt with it quite a bit in various projects, most recently my TV Library plugin for boxee. I use PHP to accomplish most of my screen scraping and its got a pretty great arsenal. However, if ease of use and simplicity is your goal, the built in tools and techniques won't be much help, they can be pretty convoluted and confusing. That's why I'm going to recommend a library called phpQuery.

phpQuery is, among other things, a PHP port of the JavaScript library jQuery. You front-end developers out there may already know where I'm going with this, but in-case you don't know, jQuery is known for many things, but is probably best known for its incredible support of CSS Selectors. CSS Selectors allow you to select HTML elements using the same syntax you'd use to style elements in a CSS Stylesheet. (Some of you more experienced readers will no-doubt be jumping to the comments section to complain about how I'm not even mentioning XPath, well I think XPath is overly complicated and can be extremely confusing to beginners. CSS Selectors are far more approachable. Also, I am of the mindset that if at anytime we can standardize on some type of well-tested technique in web development, we should.)

There could be entire books written about what CSS Selectors are and how to utilize them best, so I'm not going to go into too much detail here. If you want to learn about all the intricacies of CSS and jQuery style selectors, first look here: jQuery Docs: Selectors, and if you're still confused, a simple Google search will likely yield all the information you'll need. But for now all you need to know is this: this is the easiest way to screen scrape anything. Ever. phpQuery will let you turn this:


',$start) + 8;

	$table = substr($content,$start,$end-$start);

	preg_match_all("||U",$table,$rows);
	foreach ($rows[0] as $row){
	    if ((strpos($row,'|U",$row,$cells);
	        $number = strip_tags($cells[0][0]);
	        $name = strip_tags($cells[0][1]);
	        $position = strip_tags($cells[0][2]);
	        echo "{$position} - {$name} - Number {$number} 
\n";
	    }
	}
?>

Into this:


 tbody > tr'] as $player) {
			$player = pq($player)->find('td')->getStrings();
			print "Player: #" . $player[0] . " - " . $player[1] . " - Position: " . $player[2] . "
\n";
		}
	}
?>

Much nicer right? Yeah I thought so too. Give it a shot and let me know how it goes for you in the comments.

iPhone MySpace Web Application Preview 002

Jake Marsh — Fri, 18 Apr 2008 05:25:18 +0000

In this second screencast, I demo more of the features of my iPhone MySpace Web Application. Cover this time is listening to music from bands on MySpace as well as photo albums, and much more. (You're going to want to click the full screen button on this one, so you can see it all in detail.)

iPhone MySpace Preview Vid

Jake Marsh — Sat, 05 Apr 2008 05:12:35 +0000

Here's a quick vid demoing my new iPhone MySpace web application. (Make sure you click the fullscreen button in the top-right to see everything best)

Jake Marsh Is Awesome. OMFG. » Programming

Screen-Scraping With CSS Selectors in PHP

iPhone MySpace Web Application Preview 002

iPhone MySpace Preview Vid