Logo by rathinagiri - Contribute your own Logo!

END OF AN ERA, FRACTALFORUMS.COM IS CONTINUED ON FRACTALFORUMS.ORG

it was a great time but no longer maintainable by c.Kleinhuis contact him for any data retrieval,
thanks and see you perhaps in 10 years again

this forum will stay online for reference
News: Visit the official fractalforums.com Youtube Channel
 
*
Welcome, Guest. Please login or register. October 28, 2020, 04:36:31 AM


Login with username, password and session length


The All New FractalForums is now in Public Beta Testing! Visit FractalForums.org and check it out!


Pages: [1]   Go Down
  Print  
Share this topic on DiggShare this topic on FacebookShare this topic on GoogleShare this topic on RedditShare this topic on StumbleUponShare this topic on Twitter
Author Topic: Notification about Database Consumption  (Read 1717 times)
0 Members and 1 Guest are viewing this topic.
cKleinhuis
Administrator
Fractal Senior
*******
Posts: 7044


formerly known as 'Trifox'


WWW
« on: November 27, 2010, 02:16:34 PM »

hello all,

we have now reached a solid ~1000 posts a month, if every post would consist of 1 char, it would mean that each month 1kb of data is generated, but it is
far more, its around 500-1000kb a month, the message space in the database is right now consuming 64MB ( !! ) of space, this means that at the beginning
of the new year, a new database has to be set up, closing this whole content down into an archive like state,  i am a bit scared about that, and i hope i can
deal with it, i can run up to 10 databases with 100mb size limits, so transfer/archiving is inevitable.... i think it will reach its "full" stage early next year !

just for letting you know that the forum will be down for a couple of days when i transfer it to its archive state....
Logged

---

divide and conquer - iterate and rule - chaos is No random!
jehovajah
Global Moderator
Fractal Senior
******
Posts: 2749


May a trochoid in the void bring you peace


WWW
« Reply #1 on: November 27, 2010, 04:12:41 PM »

Thanks for that Trifox. Hope everything goes smoothly for you.

Thanks for hosting and managing a great site, too! Really a wonderful place to be and think and wonder!
Logged

May a trochoid of ¥h¶h iteratively entrain your Logos Response transforming into iridescent fractals of orgasmic delight and joy, with kindness, peace and gratitude at all scales within your experience. I beg of you to enrich others as you have been enriched, in vorticose pulsations of extravagance!
ker2x
Fractal Molossus
**
Posts: 795


WWW
« Reply #2 on: November 27, 2010, 05:03:48 PM »

If you're using postgresql, feel free to ask me for any kind of help. I'm postgresql DBA (think about hundreds of GB and hundreds of millions of row) smiley
Logged

often times... there are other approaches which are kinda crappy until you put them in the context of parallel machines
(en) http://www.blog-gpgpu.com/ , (fr) http://www.keru.org/ ,
Sysadmin & DBA @ http://www.over-blog.com/
cKleinhuis
Administrator
Fractal Senior
*******
Posts: 7044


formerly known as 'Trifox'


WWW
« Reply #3 on: November 27, 2010, 07:42:25 PM »

its the mysql database provided by my hoster, but i read now that i can run up to 20 100MB databases wink
so, the only work is to lock old/really old postings and transfer them to a duplicate read only database,
that would then be served through a different domain, and i thus not searchable by the forum ...
but as i said, it is likely to emerge in the beginning of the next year...
Logged

---

divide and conquer - iterate and rule - chaos is No random!
Softology
Conqueror
*******
Posts: 120


« Reply #4 on: November 28, 2010, 06:28:14 AM »

Two quick suggestions...

Locking older threads in an archive is OK, but...

1. If possible ensure links to the older threads will still work.  Otherwise the multitude of links to FF out there will suddenly get a 404 error.

2. There are a lot of threads/questions I have seen here that stay dormant for months or more before a new visitor repsonds and awakens them.  What would happen with the archived threads?  If there was an automatic final post in each locked/archived thread that said "Click here to continue to the rest of this thread" that pointed to a new unlocked thread that allowed continued replies it would be great.  Start a new thread with a topic like 'continuation of topic "blah etc etc"' to continue the discussion.  The first post of the continued thread could bean automated "Click here to see the closed thread this one came from".

I have no idea how difficult it would be to implement these on the current forum software, but there is so much great info within these forums that the topics and threads need to be preserved from start to finish (for new and old visitors and members alike).

Jason.
Logged
miner49er
Safarist
******
Posts: 82


« Reply #5 on: November 28, 2010, 10:50:06 PM »

Glad to hear this is a popular place!

Well, here's a few more bytes to add to the collection!
Logged
jehovajah
Global Moderator
Fractal Senior
******
Posts: 2749


May a trochoid in the void bring you peace


WWW
« Reply #6 on: November 29, 2010, 07:25:10 AM »

Two quick suggestions...

Locking older threads in an archive is OK, but...

1. If possible ensure links to the older threads will still work.  Otherwise the multitude of links to FF out there will suddenly get a 404 error.

2. There are a lot of threads/questions I have seen here that stay dormant for months or more before a new visitor repsonds and awakens them.  What would happen with the archived threads?  If there was an automatic final post in each locked/archived thread that said "Click here to continue to the rest of this thread" that pointed to a new unlocked thread that allowed continued replies it would be great.  Start a new thread with a topic like 'continuation of topic "blah etc etc"' to continue the discussion.  The first post of the continued thread could bean automated "Click here to see the closed thread this one came from".

I have no idea how difficult it would be to implement these on the current forum software, but there is so much great info within these forums that the topics and threads need to be preserved from start to finish (for new and old visitors and members alike).

Jason.

It is a concern, jason. Trifox pointed out to me once that attachments might get lost so some like me who started posting immages through attachment may need to look a adjusting their posting.

As long as the informationand links are preserved this valuable resource will suffer no injury in archiving, but my experience of other archived collections is the 404 error to links.

It is unreasonable to expect Trifox to do more than his best to preserve such a vast network of information over his christmas holiday break, so i think individual posters may want to archive their posts and use them to restore, or answer future queries.

I think Opera unite may be a useful browser service to have in this regard, it makes for good collaboration in information preservation.
Logged

May a trochoid of ¥h¶h iteratively entrain your Logos Response transforming into iridescent fractals of orgasmic delight and joy, with kindness, peace and gratitude at all scales within your experience. I beg of you to enrich others as you have been enriched, in vorticose pulsations of extravagance!
cKleinhuis
Administrator
Fractal Senior
*******
Posts: 7044


formerly known as 'Trifox'


WWW
« Reply #7 on: November 29, 2010, 11:40:30 AM »

it is most likely that all archived documents will be run on a subdomain, like "archive1.fractalforums.com" where all links linking to "fractalforums.com/..." can be repaired
through "archive1.fractalforums.com" this would be the only solution i could think of, but since the "new" forum will hold a blank new forum, it is like that the
archive will run consistently on its own, meaning all links that are relative would work, BUT, external links wont work anymore, but could be easily "repaired" in one or antoher way we will go into problems that might only be solved if the "NEW" site would go into a subdomain, so that external links keep working, or i wont do it
using a subdomain, rather than a sub folder, e.g. "fractalforums.com/new/" would be the new main page ... but i am not sure about how to do it best and consistently,
any ideas are appreciated !

i could think of a little hack into the forums, such way that if a post isnt found, it would redirect it to the "archive1" site, such that the user dont get it what happens, he just is redirected to a read only forum ...

i hope that i can rely on human brain that things will go smoothly ...

Logged

---

divide and conquer - iterate and rule - chaos is No random!
hobold
Fractal Bachius
*
Posts: 573


« Reply #8 on: November 29, 2010, 08:21:51 PM »

The links might be preserved with a little cheat:

1. The "old" server is getting locked (read-only) as planned, but retains its domain name.
2. The "new" server is getting a new name, e.g. http://newera.fractalforums.com/...
3. The front page of the "old" server is changed to redirect all browsers to the "new" server, which is blank and writable.

All deep links to the "old" server would continue to work. The standard domain name "fractalforums.com" would still send people over to the new server. The new server would need a few back links to the old server to enable searching in the archives (stuff that used to be on the old front page). And in the long run, different eras of fractalforums.com would end up having different URLs, but all URLs would be permanent.
Logged
cKleinhuis
Administrator
Fractal Senior
*******
Posts: 7044


formerly known as 'Trifox'


WWW
« Reply #9 on: November 29, 2010, 11:34:11 PM »

i checked the size of a monthly posting, it seems to settle at around 1mb message size in the database, wink,per month - this seems to be enough for more than a halve year...
the complete forum would only be made write-only, search functionality could be easily linked to the search page

what i dont like about this setup is that the "new" era would need a new domain sad but i belive a set up like this would serve the best
Logged

---

divide and conquer - iterate and rule - chaos is No random!
twinbee
Fractal Fertilizer
*****
Posts: 383



WWW
« Reply #10 on: December 04, 2010, 03:03:55 PM »

Trifox, is there a way to increase the 100mb limit for the database size to something like a gig? Maybe if you ask the server admins, they'll increase it. It would keep everything unified and save a bit of hassle! If they don't allow it, I would change host to allow for the extra host DB storage, otherwise you'll have the same problem again at 200mb etc.
« Last Edit: December 04, 2010, 03:05:57 PM by twinbee » Logged
Pages: [1]   Go Down
  Print  
 
Jump to:  

Related Topics
Subject Started by Replies Views Last post
Damien Hirst and the temple of mass consumption Mandelbulb3D Gallery Sitting Duck 2 480 Last post August 20, 2013, 11:32:34 PM
by Sitting Duck
auto email notification - user cleanup over! Fractal Forums News cKleinhuis 2 833 Last post September 02, 2013, 10:54:18 PM
by cKleinhuis
Formula database? feature request 0Encrypted0 4 855 Last post April 13, 2015, 08:24:21 PM
by 0Encrypted0
short server downtime - database full of logs Fractal Forums News cKleinhuis 1 826 Last post May 09, 2016, 09:55:50 AM
by stardust4ever
Sound Notification Feature Requests CCV 2 416 Last post April 18, 2017, 02:53:15 AM
by CCV

Powered by MySQL Powered by PHP Powered by SMF 1.1.21 | SMF © 2015, Simple Machines

Valid XHTML 1.0! Valid CSS! Dilber MC Theme by HarzeM
Page created in 0.158 seconds with 28 queries. (Pretty URLs adds 0.007s, 2q)