Welcome Guest   ·   Login   ·   Register
Forum Home  >  Add-on support  >  XMLGrab  >  Thread
Search  
  
  Advanced Search
   
 
How to handle grabbing changing to items in an XML feed
 
Jim Pannell
Posted: 12 November 2009 07:32 PM   [ Ignore ]  
Newbie
Rank
Total Posts:  11
Joined  2009-08-18

Hi Andrew

Thanks again for producing these fab addons - I’m using them a lot and finding them invaluable.

Here’s a scenario for you:

I have a property site where the properties that are displayed come from different sources: the owner adds some manually via the EE CP (although hasn’t yet!), and the others are imported via XMLGrab from external xml feeds.

The problem:
If items in the external feeds are updated, they aren’t updated when XMLGrab is re-run, and the data in EE is therefore out of date (think prices etc which are important on a property site).

My workaround:
What I’m doing currently is removing all properties before I run XMLGrab again, like this:

global $DB;

// Remove all entries within Properties weblog
$query = $DB->query("DELETE FROM `exp_weblog_data` WHERE `weblog_id` = 5");
$query = $DB->query("DELETE FROM `exp_weblog_titles` WHERE `weblog_id` = 5"); 

Of course this won’t work when the owner starts adding properties himself. I suspect I’m have a ‘source’ custom field and only delete those where the source equals ‘xml’. Any idea what my SQL would be like to do that?

What do you think?
The site isn’t live yet and I just wanted to pick your brains on whether you think this could be done better. I’ll be wanting users to be able to save certain properties on the site so it’s very important that the urls and reference numbers (not entry_id) stay the same on every import. I think it should work, any thoughts?

Cheers


Jim

Profile
 
 
Andrew Weaver
Posted: 16 November 2009 10:26 AM   [ Ignore ]   [ # 1 ]  
Administrator
RankRankRankRank
Total Posts:  151
Joined  2006-04-12

Hi Jim,

I’m working on versions of the plugins that can handle updates as well as just adding new entries. Let me know if you want to test them for me.

Andrew

Profile
 
 
Jim Pannell
Posted: 16 November 2009 10:30 AM   [ Ignore ]   [ # 2 ]  
Newbie
Rank
Total Posts:  11
Joined  2009-08-18

Hi Andrew

Yes please! Any idea of when they might be ready for testing / use? Feel free to email with the details if you don’t want to post on here.

Cheers


Jim

Profile
 
 
rorourke
Posted: 18 December 2009 05:20 PM   [ Ignore ]   [ # 3 ]  
Newbie
Rank
Total Posts:  2
Joined  2009-12-18
Andrew Weaver - 16 November 2009 10:26 AM

Hi Jim,

I’m working on versions of the plugins that can handle updates as well as just adding new entries. Let me know if you want to test them for me.

Andrew

Sorry, I have been asking you this question in all the wrong places today (comments & contact form). I am up against a very similar issue. I would love to help test when you have this feature ready…any chance you could drop a hint of when that might be?

Cheers.

Profile
 
 
Andrew Weaver
Posted: 18 December 2009 08:44 PM   [ Ignore ]   [ # 4 ]  
Administrator
RankRankRankRank
Total Posts:  151
Joined  2006-04-12

I’m hoping they should be ready for EE1.6 and EE2.0 very soon, with luck next week. Please get back to me if you don’t hear anything soon.

Thanks,
Andrew

Profile
 
 
Nick Oliver
Posted: 21 February 2010 05:04 PM   [ Ignore ]   [ # 5 ]  
Newbie
Rank
Total Posts:  1
Joined  2010-02-21

Hi Andrew,

Thanks for the great plug-in.

I was wondering if you are still working on versions of this plug-in that will update as well as insert? For both 1.6 and 2.0?

If you are still looking for testers, I’m game.

Thanks again,
Nick

Profile
 
 
Jim Pannell
Posted: 05 March 2010 10:49 AM   [ Ignore ]   [ # 6 ]  
Newbie
Rank
Total Posts:  11
Joined  2009-08-18

Hey Andrew,

So I now have XMLGrab doing it’s thing very nicely. I’m using version 0.8 with the update functionality built in.

However, I’m seeing that after an initial import, when certain data in the xml file I’m importing has changed, it isn’t updating in EE. One specific example of this is that I have 10 image fields in my feed and 10 custom fields for these images in EE. On my initial import, only 1 of these image fields (the first one) was populated. That data was carried over into EE no problem - great. But now all 10 image fields have data in them but when I re-run the xmlgrab template, none of the extra data is coming into EE.

Is this expected behavior or am I doing something wrong?

Here’s my template code:

{exp:xmlgrab url="http://www.mydomain.com/xml-feed.php" 
    itempath="/properties/property"
    weblog="5"
    author="1"
    title="property_ref"
    date="date_listed"
    use="propertyId|property_status|property_title|location_region|location_dept|location_town|location_postcode|location_nearto|property_price|property_preview|property_description|property_type_id|property_datebuilt|property_habitable_space|property_bedrooms|property_bathrooms|property_plot_size|property_land_size|property_feature_floors|property_feature_fireplaces|property_feature_garden|property_feature_terrace|property_feature_exposure|property_feature_pool|property_feature_garage|property_feature_nearshops|property_feature_nearschools|property_feature_neartransport|property_feature_neargolf|property_feature_nearcoast|property_feature_oceanview|property_feature_lakeview|property_feature_mountainview|image01|image01_caption|image02|image02_caption|image03|image03_caption|image04|image04_caption|image05|image05_caption|image06|image06_caption|image07|image07_caption|image08|image08_caption|image09|image09_caption|image10|image10_caption"
    fields="feed_propertyId|property_status|property_title|property_region|property_department|property_city|property_postcode|property_location_near-to|property_price|property_preview|property_description|property_type|property_date-built|property_area|property_bedrooms|property_bathrooms|property_plot-size|property_land-size|property_feature_floors|property_feature-fireplaces|property_feature_garden|property_feature_terrace|property_feature_exposure|property_feature_pool|property_feature-garage|property_feature_nearshops|property_feature-near-schools|property_feature-near-transport|property_feature-near-golf|property_feature_near-coast|property_feature_ocean-view|property_feature_lake-view|property_feature_mountain-view|property_image1|property_caption1|property_image2|property_caption2|property_image3|property_caption3|property_image4|property_caption4|property_image5|property_caption5|property_image6|property_caption6|property_image7|property_caption7|property_image8|property_caption8|property_image9|property_caption9|property_image10|property_caption10"
    category_field="source"
    category_group="5"
    unique="feed_propertyId,title"
    update="true"} 

Apologies for the length of the template!

Profile
 
 
Andrew Weaver
Posted: 09 March 2010 10:55 AM   [ Ignore ]   [ # 7 ]  
Administrator
RankRankRankRank
Total Posts:  151
Joined  2006-04-12

Hi Jim,

Your template looks correct and I’d expect the missing fields to be filled when you do an update.

Is anything updating, or is it just the previously empty fields that are not working?

Andrew

Profile
 
 
Jim Pannell
Posted: 12 March 2010 10:55 AM   [ Ignore ]   [ # 8 ]  
Newbie
Rank
Total Posts:  11
Joined  2009-08-18

Hi Andrew

I’m about to test this, but wanted to hear what behaviour *should* happen from the horses mouth so to speak.

What happens to entries that have been added manually to the same weblog in EE once the XMLGrab script is run? Are they deleted or ignored by the script?

Also, how would one deal with a one time import of some items and then keep them in EE on subsequent runs of XMLGrab?

Cheers


Jim

Profile
 
 
Andrew Weaver
Posted: 12 March 2010 11:03 AM   [ Ignore ]   [ # 9 ]  
Administrator
RankRankRankRank
Total Posts:  151
Joined  2006-04-12

Hi Jim,

XMLGrab will not delete any entries.

The plugin will go through the entries in the XML file and will add any that do not exist.

If you have the new version of the plugin that allows updates, then:

if the update=“true” parameter is used, it will update any entries that already exist (whether previously imported or entered manually)

if the parameter is not used, it will just ignore them and leave them as they were

Andrew

Profile
 
 
Jim Pannell
Posted: 12 March 2010 11:05 AM   [ Ignore ]   [ # 10 ]  
Newbie
Rank
Total Posts:  11
Joined  2009-08-18

Thanks for the very speedy response Andrew.

That’s exactly what I wanted to hear - phew!

Profile
 
 
David
Posted: 19 May 2010 09:01 AM   [ Ignore ]   [ # 11 ]  
Newbie
Rank
Total Posts:  1
Joined  2010-05-19

Hi Andrew

I’m in an identical situation to Jim, as he outlined in his first post.
I’m gathering from the back and forth that version 0.8 went some way to solving Jim’s problem.

Any chance I could try out the new version as well?

Thanks

Profile
 
 
   
 
 
‹‹ XMLGrab 0.7.6 published      Out of memory / possible bug ? ››