Menu
What's new
By:
Filters

KavaLounge posts

kavadude

kavadude

❦ॐ tanuki tamer
Had a cursory look at moving over the old posts. It's gonna be a little harder than I thought.

(1) Yuku doesn't provide any way to export your data that I can find, so even if the kavasseur gets fiber optic internet in Ghana there's no easy way to get at it. Gonna have to scrape it through the web which is never fun.

(2) I didn't realize XenForo was paid software. Also looks like their dev documentation is nonexistent from reading their forums. So that'll be trickier.

Anyway I'm not going to have time to take a serious look at it until the weekend. Calvincs or anyone else if you want to take a crack at it, it's all yours.
 
C

Calvincs

Kava Curious
Ill take a look at it and see what can be done.
I could create some standardized extract w/ BeautifySoap, crawl the forums, save to xml or some easily parsed document type. Then we could work on how to upload the content.
If we dont have backend db access, that could be harder.
Do you know if we have db access to the old forum?
 
The Kap'n

The Kap'n

The Groggy Kaptain (40g)
KavaForums Founder
That's the problem, the only one with that information is Doug LaRose/The Kavasseur and getting ahold of him has proven quite.....difficult. I believe he is living/working in an undeveloped part of africa where Internet Access is an extreme luxury.
 
kavadude

kavadude

❦ॐ tanuki tamer
Pretty much everything I have read suggests that even the forum owner can't export the data so yeah, it has to be scraped. I created my own yuku forum and I can't find a way to do it so even if we can get a hold of him no dice.
 
C

Calvincs

Kava Curious
How about access to the db for kavaforums.com?

Another issue we have to look at is user migration. In order to bring over post from other users, they will likely need an account created on this forum.
This would not be so bad, if we had access to their email accounts. We could create some artificial password(s) for each user, and when they came here, they could do a password recovery and get a new password.
However, after looking at the old forum, I cant 'yet' find an easy way to get access to users email accounts.
There are some ways we could get around this, but there not very pretty.
1. We create the account(s) as mentioned above, but have the users come to us and we do some sort of validation.
Example:
-->They go to a web page, put in their username and email address.
We email them a code which they need to post on the old forum. (A thread specifically for this)
We validate the code, automatically/a program does this for us. (doing this by hand would be crazy)
If valid, we update the new forum db with their email address for their imported user name. And send them a password reset request to their given email address.
-They now have access.
By having access to the old forum, we can validate existing users.

The other option is to create an archive of the old forum, that can still be accessed, but is no longer the 'official' forum.
And have users come here and create their accounts.
But anyone could take another user login name. Even if not intentionally.

Just my thoughts..
Comments, ideas?
 
The Kap'n

The Kap'n

The Groggy Kaptain (40g)
KavaForums Founder
You can have access to whatever you need on kavaforums.com just message me I'll set up the account.

I know there's a plugin for this forum which would allow a site admin to post as a user without interaction with the passwords if we needed that I could supply it.
 
kavadude

kavadude

❦ॐ tanuki tamer
I'm thinking the old posts should just go in a read-only archive forum and we don't worry about the user accounts. Much less messy. The important thing is that people can search the old posts since there's some good info in there.
 
kavadude

kavadude

❦ॐ tanuki tamer
I found a script for converting Yuku to phpbb/smf forums. Xenforo has a phpbb importer. I'm going to go ahead and try to run it against a local copy of phpbb. If nothing else hopefully we will have all the kavalounge posts in a usable format.
 
C

Calvincs

Kava Curious
kavadude said:
I found a script for converting Yuku to phpbb/smf forums. Xenforo has a phpbb importer. I'm going to go ahead and try to run it against a local copy of phpbb. If nothing else hopefully we will have all the kavalounge posts in a usable format.
Click to expand...
Where is this script located? I would like to take a look.
 
C

Calvincs

Kava Curious
I looked over the code, seems pretty straight forward.. Would love to know how well it did the expport/import into phpBB2 or SMF. Hopefully it will then be pretty easy to import into this forum.
Also, if the Python gives you any problems, let me know. I love python! Its my favorite scripting lang and do alot of work in it. So if it gives you any fuss, letme know.
 
kavadude

kavadude

❦ॐ tanuki tamer
Okay. I threw together a scraper this afternoon. I have the KavaLounge posts sitting on my hard drive in what should be a usable format. Getting them into XenForo and formatting Yuku's garbage into something usable is a different problem.
 
infraredz

infraredz

BULA!
At least you got it scraped. The Yuku boards seemed like the old 90s angelfire websites compared to our new xenforo site.

YAY for progress!
 
C

Calvincs

Kava Curious
Kavadude, is this the HTML for each page? Aka, a straight copy of each and every page.
Or was it a parsed copy of every page. It sounds like the latter..
If the latter, we could write a script to parse it into the phpBB, then use the converter to bring it over.
Ill have more time tomorrow as well. Tonight is a bit busy for me.
Let me know.
 
kavadude

kavadude

❦ॐ tanuki tamer
Yeah I parsed it and dumped it into a pickle file. The content of the posts is still HTML however; it may or may not need some work to display on the forums.

I'm also not sure that the posts are 100% harvested correctly. I was pretty exhausted by the time it finished so I haven't looked through the data extensively.
 
kavadude

kavadude

❦ॐ tanuki tamer
Almost there. I decided to go ahead and shove the posts in phpbb because it's simpler and better-documented than xenforo. But first I have to convert them all from html to bbcode, a process which is almost complete.
 
You must log in or register to reply here.
Top