Notification about Database Consumption

[cKleinhuis]

hello all,

we have now reached a solid ~1000 posts a month, if every post would consist of 1 char, it would mean that each month 1kb of data is generated, but it is
far more, its around 500-1000kb a month, the message space in the database is right now consuming 64MB ( !! ) of space, this means that at the beginning
of the new year, a new database has to be set up, closing this whole content down into an archive like state,  i am a bit scared about that, and i hope i can
deal with it, i can run up to 10 databases with 100mb size limits, so transfer/archiving is inevitable.... i think it will reach its "full" stage early next year !

just for letting you know that the forum will be down for a couple of days when i transfer it to its archive state....

[jehovajah]

Thanks for that Trifox. Hope everything goes smoothly for you.

Thanks for hosting and managing a great site, too! Really a wonderful place to be and think and wonder!

[ker2x]

If you're using postgresql, feel free to ask me for any kind of help. I'm postgresql DBA (think about hundreds of GB and hundreds of millions of row)

[cKleinhuis]

its the mysql database provided by my hoster, but i read now that i can run up to 20 100MB databases
so, the only work is to lock old/really old postings and transfer them to a duplicate read only database,
that would then be served through a different domain, and i thus not searchable by the forum ...
but as i said, it is likely to emerge in the beginning of the next year...

[Softology]

Two quick suggestions...

Locking older threads in an archive is OK, but...

1. If possible ensure links to the older threads will still work.  Otherwise the multitude of links to FF out there will suddenly get a 404 error.

2. There are a lot of threads/questions I have seen here that stay dormant for months or more before a new visitor repsonds and awakens them.  What would happen with the archived threads?  If there was an automatic final post in each locked/archived thread that said "Click here to continue to the rest of this thread" that pointed to a new unlocked thread that allowed continued replies it would be great.  Start a new thread with a topic like 'continuation of topic "blah etc etc"' to continue the discussion.  The first post of the continued thread could bean automated "Click here to see the closed thread this one came from".

I have no idea how difficult it would be to implement these on the current forum software, but there is so much great info within these forums that the topics and threads need to be preserved from start to finish (for new and old visitors and members alike).

Jason.

[miner49er]

Glad to hear this is a popular place!

Well, here's a few more bytes to add to the collection!

[jehovajah]

Two quick suggestions...

Locking older threads in an archive is OK, but...

1. If possible ensure links to the older threads will still work.  Otherwise the multitude of links to FF out there will suddenly get a 404 error.

2. There are a lot of threads/questions I have seen here that stay dormant for months or more before a new visitor repsonds and awakens them.  What would happen with the archived threads?  If there was an automatic final post in each locked/archived thread that said "Click here to continue to the rest of this thread" that pointed to a new unlocked thread that allowed continued replies it would be great.  Start a new thread with a topic like 'continuation of topic "blah etc etc"' to continue the discussion.  The first post of the continued thread could bean automated "Click here to see the closed thread this one came from".

I have no idea how difficult it would be to implement these on the current forum software, but there is so much great info within these forums that the topics and threads need to be preserved from start to finish (for new and old visitors and members alike).

Jason.

It is a concern, jason. Trifox pointed out to me once that attachments might get lost so some like me who started posting immages through attachment may need to look a adjusting their posting.

As long as the informationand links are preserved this valuable resource will suffer no injury in archiving, but my experience of other archived collections is the 404 error to links.

It is unreasonable to expect Trifox to do more than his best to preserve such a vast network of information over his christmas holiday break, so i think individual posters may want to archive their posts and use them to restore, or answer future queries.

I think Opera unite may be a useful browser service to have in this regard, it makes for good collaboration in information preservation.

[cKleinhuis]

it is most likely that all archived documents will be run on a subdomain, like "archive1.fractalforums.com" where all links linking to "fractalforums.com/..." can be repaired
through "archive1.fractalforums.com" this would be the only solution i could think of, but since the "new" forum will hold a blank new forum, it is like that the
archive will run consistently on its own, meaning all links that are relative would work, BUT, external links wont work anymore, but could be easily "repaired" in one or antoher way we will go into problems that might only be solved if the "NEW" site would go into a subdomain, so that external links keep working, or i wont do it
using a subdomain, rather than a sub folder, e.g. "fractalforums.com/new/" would be the new main page ... but i am not sure about how to do it best and consistently,
any ideas are appreciated !

i could think of a little hack into the forums, such way that if a post isnt found, it would redirect it to the "archive1" site, such that the user dont get it what happens, he just is redirected to a read only forum ...

i hope that i can rely on human brain that things will go smoothly ...

[hobold]

The links might be preserved with a little cheat:

1. The "old" server is getting locked (read-only) as planned, but retains its domain name.
2. The "new" server is getting a new name, e.g. http://newera.fractalforums.com/...
3. The front page of the "old" server is changed to redirect all browsers to the "new" server, which is blank and writable.

All deep links to the "old" server would continue to work. The standard domain name "fractalforums.com" would still send people over to the new server. The new server would need a few back links to the old server to enable searching in the archives (stuff that used to be on the old front page). And in the long run, different eras of fractalforums.com would end up having different URLs, but all URLs would be permanent.

[cKleinhuis]

i checked the size of a monthly posting, it seems to settle at around 1mb message size in the database, ,per month - this seems to be enough for more than a halve year...
the complete forum would only be made write-only, search functionality could be easily linked to the search page

what i dont like about this setup is that the "new" era would need a new domain but i belive a set up like this would serve the best

[twinbee]

Trifox, is there a way to increase the 100mb limit for the database size to something like a gig? Maybe if you ask the server admins, they'll increase it. It would keep everything unified and save a bit of hassle! If they don't allow it, I would change host to allow for the extra host DB storage, otherwise you'll have the same problem again at 200mb etc.