Product Feed Integration and Scraping Products from Supplier Web Sites

Mar 5, 2008 · 3 minute read
Category: spidering ecommerce product catalogue product feed scraping

This is post is now quite old and the the information it contains may be out of date or innacurate.

If you find any errors or have any suggestions to update the information please let us know or create a pull request on GitHub

One of the big tasks that any ecommerce retail business must undertake is the continual updating and inserting of products into the catalogue. Done one by one, this task can take a ridiculous amount of time. In some instances there is no better option, but in the vast majority of cases there is!

Product Feed

The ideal scenario is that your supplier makes available an up to date product feed which is regularly updated and contains all of the information you need to insert those products into your catalogue. The challenge with this is that it is highly unlikely that you will literally be able to upload this data as is. The reason being that each ecommerce system has its own quirks and separate ways of doing things. Before you can upload this data into your feed, it is highly likely that it will need to be altered and prepared for insertion.

You could do updating by hand - but that brings us back to our first point. Doing things by hand can take a ridiculously large amount of time. Instead - we recommend that you have a script which does all this preparation for you.

In fact this task is something that Edmonds Commerce specialise in. Not least because it is something that we have done plenty of and so we have a good understanding of how to do the job. Furthermore - we understand how to do the job well.

Spidering and Scraping Products from Supplier Web Site

If your supplier does not provide a feed, or if the feed they supply does not have all of the information that you want, you might think you are stuck. You are not!

It is perfectly possible to build a system which will visit every product on your suppliers web site and grab all of the information and pictures and then save them into a format that you can insert into your catalogue system. It is even possible to extend the scraping system so that it goes all the way and inserts the products into your site for you.

Again this is something that Edmonds Commerce specialise in.

Conclusion

If you find that you or your staff are spending large amounts of time manually copying and pasting information from supplier web sites - you need to ask yourself if that is really cost effective. Whilst developing a script to process a feed or scrape a supplier web site might involve a significant initial outlay - the humongous saving in staff time ensures that you will quickly recoup this cost and then will be straight into a profitable scenario. Furthermore your catalogue will be absolutely up to date with the latest pictures, information and prices meaning that you have the best chance to sell those products.

If you want to discuss how Edmonds Commerce could help you achieve these great goals of cost reduction and totally up to date catalogue - please do get in touch.