Help Wikitravel grow by contributing to an article! Learn how.

Wikitravel talk:Database dump

From Wikitravel
Jump to: navigation, search

Project forks[edit]

Actually, I think it is fair to mention Wikivoyage specifically here because the failure to provide database dumps, and the lack of even an explanation for this failure, is a good faith issue which drains off contributors to that site in particular, thereby weakening our contributor base and generally diluting the strength of the "free-content travel guide community" (if I can call it that?). --Peter Talk 00:54, 4 September 2007 (EDT)

For reference see Wikitravel talk:How to re-use Wikitravel guides#Keeping an up to date mirror and Wikitravel talk:How to re-use Wikitravel guides#Data Dumps? for older discussions on the subject. Also see Wikitravel:How to re-use Wikitravel guides#XML data feeds, which claims it is possible to get a full XML dump of Wikitravel, although to my knowledge no one has ever received one. I suspect shared: has additional discussions on the subject, but didn't search for them. -- Ryan • (talk) • 01:13, 4 September 2007 (EDT)
I agree that there should be dumps. I removed the WV link as part of an ongoing effort to keep their spam off of our site. – cacahuate talk 01:19, 4 September 2007 (EDT)
I think it's antagonizing to consider mentioning a related project as spam. Especially if the project is very relevant in this context, since it is offering database dumps.
Also note that I haven't given up on contributing to WT just yet. So I also consider it a bit weird to consider "me" part of "them". Guaka 10:27, 4 September 2007 (EDT)
I don't consider you part of them... but we've had a bit of a WV spam problem lately, which admittedly I have little tolerance for... the sentence as it is now is true and more general. Lack of dumps may lead to project forks... not just to WV. Personally if I left WT and wanted to fork, I would start something new, not an English version of WV... which is why I would prefer the sentence to be more broad. – cacahuate talk 03:14, 7 September 2007 (EDT)
Hi Guys I just wanted to weigh in that WV isnt the only who has forked and had some interesting dealings with XML dumps. If I went into details it would probably come across spammy or start a flame war, so basically I just leave it at we had a surprising explanation surrounding the No we received from IB. As a result, I'd be very amazed if you guys ever see data dumps on WT so try no to be too hard on folks who've forked to other projects. -- Jeremy (talk) 01:56, 12 September 2007 (EDT)
Actually, I would be quite interested in the reasons given...? Jpatokal 02:13, 13 September 2007 (EDT)
Me too, details please – cacahuate talk 02:17, 13 September 2007 (EDT)
Me three. The lack of available database dumps or even the rumored XML exports has been a sore point with contributors for a long time, although not so sore a point that most people have wanted to fork the project. What was your experience? -- Ryan • (talk) • 02:48, 13 September 2007 (EDT)


Ok I'll try to be brief and neutral: About a year ago I got back from a 9 month trip around the world, and decided to start an online travel information company (Travature). Mostly because, in my own travels I was frustrated by the lack of integration between different travel segments (flights, guides, etc) and the fact that nearly every travel company was trying to sell you something, nobody was just trying to help out travelers. I really liked the base of wikitravel, they seemed to share my view on free content for travelers. So I contacted IB to see if we could get a copy of the data and work out a partnership with my new company. We were/are CC-by-SA and so I figured a modified version of the content would augment perfectly with the what we were building, and I felt like in the long run, because we shared licenses we could equally give back to WT. However IB only wanted to give us a 250 character stream. Not the whole data. We wanted the data because we were planning on building a much different type of wiki (think wikitravel meets yelp meets kayak). We weren't trying to divide the wikitravel community, we just wanted to build something completely different. My hope was to start with wikitravel content for guides, but as Travature began to morph into a completely different animal we would continue to reupdate back into the wikitravel community what ever they/you deemed useful from us. My original plan was to prominently give WT credit and show our partnership at every opportunity. However all my partnership plans fell apart, as IB basically said that what we were doing wasn't in their interests and so they declined giving us a dump. My perspective was it wasn't "their" data to make decisions on who should and shouldn't get access to it. Anyways, in the few months since I've kept my mouth shut and my teams quietly worked on building our system, however today we finally finished the technical backend to do dumps of our own, so I felt like I could finally in good conscience weigh in on the issue. -- Jeremy (talk) 02:53, 12 September 2007 (EDT)
I have a fork of Wikitravel that we made about 2 months ago for a project that I have now decided not to persue. If anyone is interested in the project and using it to create dumps or for other creative purposes, please let me know. You can see where we started the project at: http://siteground181.com/~travelh2/wiki/index.php?title=Main_Page . We are a commercial venture, but open to any new idea with merit. Note that right now the site is locked down, but that was only because we really never did anything with it. Best Rbe2004 17:14, 8 November 2007 (EST)

Regarding use of a crawler to download content[edit]

swept in from the pub

I am working on a project in the field of data mining and machine learning in the travel domain, for which we require high-quality content to conduct some experiments. It would be great if we could have access to the data dumps. Alternatively, would it be fine if we crawl through a section of this website ? Software.research.work 08:52, 9 November 2009 (EST)

See Wikitravel:Terms of use#Spiders --inas 16:51, 9 November 2009 (EST)
If you want data dumps then use Special:Export. —The preceding comment was added by 121.73.78.167 (talkcontribs)


Hi there

Sorry to hear about your dump trouble :(

I have added some dumps to allow you guys to download it :). The dates are 14th June 2010 at 3:30 Unfortunatly the imagegetter tool didnt do a great job so most of the images are missing... I cant seem to find the source code anywhere for imagegetter.exe or who created it on the net - if anyone knows this info it would be greatly appreciated. You can email me on allebone@gmail.com (Peter Allebone)

Complete files: (only download if you really really want to) http://dl.dropbox.com/u/63233/Wikitravel/Complete%20zip/WikitravelComplete14-June-2010.7z

Just the binaries for the wiki2touch app on the iphone: http://dl.dropbox.com/u/63233/Wikitravel/Completed%20binaries%20for%20iphone/Completed%20binaries%20for%20iphone.7z

Interum dumps and logs: (the xml file for other offline wiki readers) http://dl.dropbox.com/u/63233/Wikitravel/Interum%20dumps%20and%20logs%20-%20the%20xml%20bzip%20file/Interum%20dumps%20and%20logs%20-%20the%20xml%20bzip%20file.7z

Source code and tools used: http://dl.dropbox.com/u/63233/Wikitravel/Source%20Code%20and%20tools/Source%20Code%20and%20tools.7z

I hope this helps, I will try make anouther dump next month :)

Pete - 15-06-2010

HI

I added some more dumps Please go to www.allebone.org to get the dumps.

Kind Regards Pete - 22-07-2010


Thanks for this, you rock! Emijrp 08:39, 31 October 2010 (EDT)

WikiTeam[edit]

Hi. I'm glad to announce we did dumps for every WikiTravel language. Emijrp 10:11, 3 March 2012 (EST)

WikiTeam, could you keep it up-to-date, too?

Variants

Actions

Destination Docents

In other languages