Grab Yahoo class was one of my works that has been used by many people around the world. Since I released this class I did not have a single month without comments or questions or suggestion about this class. Some people have copied the class, released it under their name (simply taking out my comments and added their own) to increase traffic to their site probably. But more importantly this class was appreciated by many. This was one of my finest contribution to the open source world.
How did I come up with this idea?
Well, I cannot remember which user group but in one of the groups I participate in had a discussion over if PHP can pull data off from places like Yahoo and use it for some purpose. Users on the group said this can be done (easily) with Java, but cannot be done with PHP. So I started researching over it. I found the cURL extension for PHP which seemed the good candidate to do the job. So I started working on it. After days of coding I finally wrote a script that will allow someone to login to their Yahoo account and fetch data from their address book. Initially I simply had a script but then I converted it to a Class allowing people to grab their address book, messenger list, number of new emails and calendar data.
What is happening now?
As per the title of this post this class is in “coma”. The reason is the update to Yahoo address book export mechanism. They have placed a CAPTCHA validation page in the export feature and, therefore, my Grab Yahoo class cannot go any further to grab the content it needs. Same thing happened with the LinkMeIn class which stopped working after LinkedIn added the CAPTCHA validation.
Why is this class in “coma”?
I have looked at the Yahoo address book export feature and found the CAPTCHA validation which will not allow the class to work. But I’ve not done my complete research on it yet, to make sure there is no other way to make this class functional. So, for the time being, this class is in “coma”. If I fail to revive this class I will probably officially pronounce it “dead”.
Can you help?
Yes, of course, you can. I hardly have time, these days, to sit with these side projects. If anyone from the community has time to research and help me revive this class it will be appreciated. Your name will go into the credit section of the class, well that is all I can offer!
After I bought my Mac Book Pro in March, one of the software I installed instantly was Yahoo! Messenger. I use it to talk to my parents everyday, who are in Bangladesh at the moment. Besides that fact, most of my open source community friends are on Yahoo! Messenger, not to mention few of my friends also.
When I installed YIM, it was kind of disappointing. It didn’t have the voice support and, therefore, I had carry my office laptop everyday with me. As Tithi uses her laptop to work on her course works, I needed to bring in office laptop so that I could talk to my parents back home. I used to constantly check Yahoo! Messenger’s blog and kept my eyes open. Finally about couple of weeks ago I found out that they have released a new build for Yahoo! Messenger for Mac 3.0 (BETA). I (literally) started jumping in joy and finally when I talked to my parents later that night, I was really happy. Hope YIM will port all the other fancy features available on Windows version, but I guess I can live with the features offered on the current version. Thank you guys (YIM Team) for making it happen. Keep up the good work.
Ever wanted to redirect users from one domain to other or one sub-domain to other without compromising the the file path or query string variables? I’ve come across the requirement when I published my WordPress blog.
For testing, I initially installed WordPress on a testing sub-domain, where I imported all the blog entries and added blog-roll links. After doing so, PHPClasses.org picked up my entries for Grab Yahoo, LinkMeIn and MySpace profile updater as track-back links. Later when I switched to my main sub-domain, I’d to find a way to keep the track-back links as is, and redirect users to the proper locations. The trick was done by .htaccess file.
After reading the apache’s documentation on mod_rewrite module, I’ve successfully implemented it. If you want people accessing http://sub1.yourdomain.com/2/12/some-post-of-yours/ to go to http://sub2.yourdomain.com/2/12/some-post-of-yours/ then in the root of sub1 (first sub-domain) create a .htaccess file and add the following lines:
RewriteEngine On
RewriteRule (.*) http://sub2.yourdomain.com/$1
I have not read much and not sure if there’s any other option to do this more efficiently; but this trick has served my purpose.
Today my new class LinkMeIn was released on PHPClasses.org. I have developed this class about couple of months back but did not have enough time to sit and arrange the files to upload on PHPClasses.org. After Grab Yahoo and MySpace Profile Updater classes this is my third class for same type. This class allows any registered LinkedIn.com user to enter their login information and get the contact list. The list is returned as an array or RSS 2.0 format.
I will update LinkMeIn class in future to add more functionalities to it. For the time being if you want to test it, please visit the LinkMeIn Demo location. To report any bug please use PHPClasses.org Support Forum and if you want to contact me directly please go to the Contact section.
I have added a new functionality for the Grab Yahoo class. Soon after I released the patch, I figured out that I could add this new functionality which did not take me much time to implement. This new feature will allow you to grab your Yahoo! Calendar in an array format.
Using Yahoo’s calendar export feature, I grab the CSV file and parse it to form an array. This was a very simple addition as it uses same method as Address Book parsing. Only thing that was different was the way I had to parse the CSV, which Yahoo creates for Outlook. I don’t have any further plan to upgrade this class but do have other plans to build similar web service based classes.
Last week suddenly my Grab Yahoo class stopped working for Address Book section. After Mr Mandy Singh reported the bug on phpclasses.org discussion board I started digging to resolve the issue.
When requesting the address book CSV from Yahoo a value for the key “crumb” is sent in the query string. Earlier Yahoo never checked for the validity of the variable but they have changed the way it is used. Now they check if the value for “crumb” is valid or not. As my script was using a predefined value and it has expired, Yahoo server could not validate. Yahoo was showing an error stating “Invalid or missing crumb”.
I modified my class so that it first loads the page from where request for the CSV is made. From the source of the page, it locates the latest value for crumb assigned by Yahoo and uses that value to send request for the address book CSV file. It is now working as expected. If you find any bug please report it on phpclasses.org discussion board.
Cool, phpResource Group now has 300 Members!! Well as I’m writing this post the group already has 13 more members registered. We hope very soon we will reach our goals with the group. Surely our intentions are to be at the top but not to beat anyone. Rather to stand on the front row.
We (moderators) are looking to provide more and more services to the group. Our current and main target now is to start the full service to our official site, www.phpresgroup.org where we will incorporate all the other activities not found within the Yahoo! Groups service.
Thanks to all our Group Members for their effortless contributions and not to mention, Thanks to the moderators.
Today my class titled Grab Yahoo has been published on www.phpclasses.org.
This class is a collation of my previous two scripts in PHP to grab Yahoo address book and messenger list. You will have to provide your
Yahoo account username and password to it which will return an array of the list of your choosen service (Address Book or Messenger List).
This will be useful in a way that anyone willing to provide the service like upon user registration to the site, you ask the person to invite people on their Yahoo address book etc. Similar to what Hi5, Name Database etc does.
First of all Thanks to Mr Arun for pointing out the problem in running the script (Yahoo! Messenger Friend List) on localhost. When I started working with cURL to grab the Yahoo address book, I faced the problem. Later when I uploaded my script on to server, it did not give any error and since then I have been testing my code there instead of testing it locally. But due to the fact that people will definitely try to run it on their localhost, I came to realisation that it is necessary to provide a solution. So after few hours of investigation I finally resolved it.
Basically the problem is that cURL is unable to verify the SSL certificate on localhost. The error that is generated by cURL is:
SSL certificate problem, verify that the CA cert is OK. Details: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
If you print the cURL error in my code, you will see this error generated on localhost.
echo curl_error($ch);
To solve this problem, first we will need to provide a file that is known as CA Bundle for cURL to read and verify the SSL certificate. Download the text format file for CA Bundle from here. On the site it says the file is in PEM format, but on my WinXP I saved the file as “CRT” file. This extension shows the correct icon. Although the extension really does not matter because all cURL will use it for is to verify the SSL certificate.
This is not the only problem that I faced while working on localhost. I found the cURL option CURLOPT_COOKIEJAR does not save the cookie file. I found a nice explanation of it on php.net. There were few options that was given by users. One of them was that after a cookie is created the cookie information reside in the header. When CURLOPT_COOKIEJAR cannot write it to a file you can write the header information using CURLOPT_WRITEHEADER to a file and grab the cookie information from there. But one of the other option solved the problem very easily. Relative path to the file that I specified was not understood by cURL. So instead I had to write full path for the filename which worked perfectly. cURL option CURLOPT_COOKIEJAR wrote the file without any problem.
The last problem that I faced was a little tricky and took most of my time solving it. I was getting authenticated by Yahoo without any problem but when I was trying to go to next step, which is actually to grab my messenger list a cURL error was generated “Can’t connect to host”. I could not understand why one part of the code connects to Yahoo and authenticates and the other cannot connect to any site at all. This was probably due to my tired brain which took this much time figuring out the problem. The problem is that I use HTTP proxy server!! But do not require any proxy to connect to secured servers. Solution to this is to use cURL option CURLOPT_PROXY to specify your proxy server address and port number, only if you require proxy server. According to my code, it should be placed in the second part of cURL operation, which is initalised to $ch1.
This small changes resolves all problems and the script works on localhost perfectly. Here is a quick look towards the change required in get_list.php file.
Step 1: Download the CA Bundle file and place it where your server can access. Make sure the file and directory has the right permission setup.
Step 2: Place the following line before performing the first cURL session, i.e. curl_exec($ch); [line number 64]
I prefer to use DOCUMENT_ROOT, but the path should be as per your choice and must be accessible by your server.
Step 4: This step is to be followed only if you are using proxy server. Specify the following line before performing your cURL session. If you require proxy for both http and secured server then the lines should be used in the both sessions.
After solving this to run on localhost, I am now interested to make the code more customisable. Use of constants, methods etc will be very nice. I also need to clean up the code a lot. Please contact me if anyone of you face problem using it either on localhost or remote server.
Using cURL in PHP is not that tough after all. Just the other day I came up with a script that grabs the Yahoo! Address Book. Today I have modified the same script to grab Yahoo! Messenger friend list.
One must know the URLs that Yahoo! goes through and data that it passes for authentication. After cURL helps you to authenticate on Yahoo, you simply write a Cookie file to contain the session information. This file is used later part of the script where cURL helps you to grab the address book or messenger list.
You will need to trick Yahoo by sending header information like HTTP Referrer. Address book is grabbed in CSV format, which needs to be parsed to show your desired information. There are 82 columns that address book sends you!! For messenger it is just cleaning up a HTML file generated by cURL. You instruct cURL not to give any output of the generated result, then save it on a temporary file, clean up unnecessary information, reformat the remaining information and print it on the screen. At the end you MUST use unlink() to delete all the files that were generated during the process. Cookie file, messenger list file etc. I have deleted them, being a good boy!!
The script for grabbing Yahoo messenger list has been published on weberdev.com. Please click “Yahoo! Messenger Friend List” to see my code on weberdev.com and download the ZIP attached to the example.
![[del.icio.us]](http://ehsan.bdwebwork.com/wp-content/plugins/bookmarkify/delicious.png)
![[Digg]](http://ehsan.bdwebwork.com/wp-content/plugins/bookmarkify/digg.png)
![[Facebook]](http://ehsan.bdwebwork.com/wp-content/plugins/bookmarkify/facebook.png)
![[Feed Me Links]](http://ehsan.bdwebwork.com/wp-content/plugins/bookmarkify/feedmelinks.png)
![[Google]](http://ehsan.bdwebwork.com/wp-content/plugins/bookmarkify/google.png)
![[LinkedIn]](http://ehsan.bdwebwork.com/wp-content/plugins/bookmarkify/linkedin.png)
![[StumbleUpon]](http://ehsan.bdwebwork.com/wp-content/plugins/bookmarkify/stumbleupon.png)
![[Technorati]](http://ehsan.bdwebwork.com/wp-content/plugins/bookmarkify/technorati.png)
![[Twitter]](http://ehsan.bdwebwork.com/wp-content/plugins/bookmarkify/twitter.png)
