Yesterday my photography blog got “farked“. This ought to have been a day for celebrating but in actual fact the first I knew about it was when someone emailed to say my site was down.
It turns out my web host has a policy where they disable a site that uses more than 20% of processor. It seems that my (pretty much default other than template) Drupal install was gobbling up resources. According to Metrics I received an additional ~8000 visitors before my blog died (or was killed, either or).
Lessons learned
- Use traffic management settings before things like this happen – in Drupal that means switch caching and throttling on and be careful what frivolous modules you have running
- Check your blog every day at least, don’t wait for a visitor to tell you that your blog is down
- You get what you pay for where hosting companies are concerned
- Read the terms and conditions before signing up
- Take regular backups of all important files and databases
- Host critical domains outside of your hosting agreement (if things get nasty you don’t want your domain held to ransom)
- It might be a hobby blog to you but people miss your stuff when it isn’t there
Luckily my photography blog is a labour of love rather than something I rely on for income. Just imagine if it was critical and I had been away on business or vacation, I might have been looking at a hole in my bank balance a week from now.
The thing that will be most interest to you guys that have been following the social bookmarking threads is out of those ~8000 additional visitors I gained one new user, 10 additional adsense clicks and zero RSS subscribers.
Just goes to show, I would much rather receive steady traffic than a flood!
Added: A couple of Metrics charts to see a little better what happened. Notice the cut-off of traffic when my blog was pulled!
Added: 23rd August, the party is over. Fun while it lasted!
30 thoughts on “Fark Killed My Blog”
If it is highly relevant rather than just to show us the site, link away.
Hi,
One of my sites got on the first page of a social site, but after 24 hours the traffic dropped exactly where it was before, so the gain was close to zero. Besides, I had this server resources problem with my hosting company (they did not suspend my site but I received about three e-mails from them, warning me to stop)
I’d like to share the story with you, but I’m not sure if it is OK to place the link in here, so I won’t.
Briefly, I agree that constant traffic is much better than spikes.
Simonne
Yep, this is good practice. Of course you can’t ever completely protect your domain, no matter how innocent you are if someone decides they have it in for you. A few dozen spam accusations to your *domain name* host from fake email addresses may sometimes result in your domain hosting being suspended.
Then you’re screwed! Forget DDOS attacks, they’re nothing compared to this happening to you. It might take days or weeks to get your domain name cleared again with your domain name host. Apparently GoDaddy have done this to some people – don’t ask me where I read that (I read a *lot*) but it was recent. Ah, hang on, I’m pretty sure it was in the Butterfly Marketing Manuscript, okay maybe not completely sure without reading it a third time.
At any rate, it’s a sobering thought that your website (and possibly income too) can disappear through no real fault of your own.
Meanwhile, here is an interesting implementation by a drupal user:
http://www.aguntherphotography.com/projects/website_cache.htm
There is a static text file cache coming soon (?) but right now there doesn’t seem to be a way to declare a page as static html versus dynamic PHP. Even with the caching options a fair amount of “core drupal stuff” needs to be loaded before you serve content, even with the paired down options (eg. “fastpath” I think).
There is no module or mode for Drupal to write static pages?
That’s the advantage of Typo3 as far as I can remember. You can design a page template to be static, dynamic or semi-dynamic.
I have no idea how Textpattern is handling this but as with every other dynamic CMS it is always a good idea to limit plugins to a minimum and to carefully check the caching options. Not to forget the PHP optimizers. I will have to put this on my To-Do list …
Well the traffic after I converted that one post to static is over double and it is humming along nicely. Static is the answer, plain HTML is good.
As markus and chris both pointed out, part of handling a /. effect is to be prepared. While each of Chris’ blocks and modules isn’t a huge problem on its own throttling them would probably have helped. Also, 4.7 has performance/cache improvements over 4.6, but maybe not enough to save this site….
Funny, this just happened to a friend of mine. His site got the #1 link at 9.yahoo.com, some community site neither of us had heard of before. He found out that GoDaddy.com has a 50 connection limit for his plan 🙁 As he has a very photo-intensive site, it was tough sledding for awhile. I pointed out to him that a more responsive site meant shorter connections, so he worked hard at cleaning up his HTML, to great effect. But he had never heard of the limit before and the customer support person admitted it wasn’t really in writing anywhere. But he also noticed almost no return traffic or ad clicking.
Yeah, the fark crowd normally will help you up again after they have ‘farked’ your site 🙂
People are so curious after the first ‘farked’ comments that they will come back to see how you react to it.
I was waiting for that update. Thanks 🙂
Update:
I was just thinking that you can ride such a wave actively and use the drop-out for some web reputation.
You see traffic coming in from fark.
Verify the referrer.
If you are really on a high position on fark, then:
Create that static landing page we spoke about
Include some message on top ‘static page because fark.com is hunting down my poor little Drupal.’ Give a backlink to the fark article.
Say something like ‘The server will be down soon. But you may a want to have a last chance to see my homepage’ and include a link to your ‘homepage’ which in fact will be a second static page including some screenshots from your homepage.
Take Drupal down but in all cases leave the prepared static pages online. If you really take Drupal down you should have a third static page in place which will officially declare your site as ‘farked’ and link to the fark article and your landing page. Oh, and redirect 404 to that third page.
Your site is available, you proof to be a real webmaster, you play the fark game actively and that way you will become famous and be invited to many TV shows 🙂
Well, a real webmaster would have a script in place which reacts automatically when a certain PI limit per minute is reached 🙂
The knock-on affect has kicked in, my post has appeard on Reddit, Digg and Del.icio.us. Looks like it was initially on stumbleupon which gave the launchpad for the others including fark. Because of these other mentions the link is being dropped in forums, am hoping blogs will pick it up from there (because I would much rather proper links than obfuscated/tinyurl/redirect links). The best bit is while these other services are not providing as much traffic as fark did I *am* getting RSS subscribers (~30) and a few more ad clicks (~15).
Markus, yes I think it peaks when the host short term terminates for processor use and shows a “please try again later” message then they pull the plug finally at the end.
…but boy, that would be sweet. I have been linked from College Humor and Fleshbot (I don’t have a porn blog! trust me!), and despite a huge surge in visits, commenting was rather flat and ad revenue was (relatively) unremarkable. I gotta believe that getting linked from AOL is about the best case scenario. Getting Farked or Slashdotted is relatively worthless in terms of ad revenue, but the benefit in that scenario is that you may be able to gain loyal readers–and hopefully, those readers are bloggers.
@underscorebleach: This is a great tip!
This has to be on a worst case scenario plan. If I find some time I have to test it with my Textpattern Installation.
@Chris: Is that flat platform at the 2.500 line due to throttling or static server limitations?
I have noticed a difference in traffic too. When I got linked form the AOL blog and was on the front page of a bunch of their pages, AOL sent me a bunch of traffic, and the traffic that came were CLICKERS. They clicked ads like crazy and commented on articles. I made more money on AdSense that day than any other day ever. CTR was over 4$.
Fark and BoingBoing sends TONS of traffic (especially Fark) but they don’t click ads as much as AOLers. I don’t mind though, if I can get some of those people coming back or telling their friends, then I’m happy. I have seen a consistently higher traffic rate even now after the Fark and BB traffic has subsided – lots more ppl commenting and participating on the site.
BB didn’t send as much traffic as Fark, but lots of ppl read BB and will link to a site mentioned on BB. I got tons of links form otehr sites thanks to the BB link. I acyually jumped over 3,000 places on Technorati in the space of a few days.
Fun stuff.
Another view…
My blog got Farked twice this month, collegehumor once, and gorillamask a couple of time. When that happens, I always generate lots of comments (maybe 40-50 for the story in question) and ad revenue increases quite a bit. eCPM does drop a bit, but the volume lifts it up (one of the Fark hits gave me nearly 100k visitors in one day).
I run moveable type, and used to serve dynamic pages, but got tired of the server going down under strain whenever it got hit by fark or digg or whoever. So I reluctantly switched to static pages, with a daily 4am crontab rebuild. No complaints from my host since then (Dreamhost), and I actually barely miss the functionality the dynamic pages gave me.
Does the traffic stay? That’s really hard to say – maybe a few. I always see those waves as Direct Mail drops – a free trial of my blog that maybe 0.5%, if I’m very lucky, will take me up on. In all the peaks, traffic definitely seems to be rising overall.
As has been said before, the best thing about making a huge site is that it will be followed in the next few days by a dribble of other blogs linking to me, who’ve picked up the story. And the real revenue is in the long tail, over the months from small numbers of traffic they all drive.
Heh, I did get some comments … not ones I would have *wanted* …
I can report similar findings on the value of traffic spikes… I’ve had blogs linked to from Fark, Collegehumor, etc and while it was fun to watch the stats counter spin, there was no direct benefit other than a possible increase in googlejuice.
College Humor once sent me 50,000 visitors over the course of three days. Of those 50K, not one commented, clicked through on ads, bought anything or subscribed. It was sort of depressing, actually.
Links from smaller, more targeted sites have proved to be much more useful, without trashing my bandwidth.
Server-side Caching shouldn’t affect javascript tracking, and I think drupal only caches certain elements of the page so most of the dynamic stuff will still be served up.
Does caching mess with pageviews or ad impressions being counted? I would guess not with javascript, which is dynamically served at all times. Just curious… this stuff is a bit beyond my confort level, but the WP caching plugin in sweet.
I’ve been linked from big sites and haven’t had too much of a problem. It’s because my blog used to be hosted on Movable Type, with static archives. (This is about the ONLY time when static archives benefited me. :/ )
Anyway, an improved means of static archiving in a pinch is to use Firefox’s “Web Page, complete” Save As. It works well for the most part (except @import’s of CSS) and sticks all images in a subdirectory.
You can then move all images and external files to another host. Then modify the saved HTML page to reference this external location (can just use a BASE tag), which will be easy because of the way FF archives to a subdirectory.
I’d combine this with Markus’ directory idea above, and you should be able to serve a LOT of static HTML pages from your host, particularly since images and such are external.
Of course, this idea doesn’t scale very well when you get Farked/Slashdotted to a site where it’s all about MULTIPLE pages. However, I gotta believe that a lot of people from these sites hit the initial, linked page and leave, so this would still help a lot.
Version = 4.6.3
Cache and Throttling – enabled
Modules – I had HOF and statistics enabled, disabled them as soon as I heard
Template – modified friendselectric theme
Blocks – all the “recent”, recent posts, recent comments, plus the all time popular posts
Comments are moderated
I really think this is more the host than drupal. My setup is pretty much based on performancing (other than perf was recently upgraded)
Chris – can you give some details on your site settings like whether the cache was enabled, which version of Drupal you are using (just 4.5, 4.6, or 4.7 – I don’t care about minor version if you don’t want to divulge it), which modules you’ve got installed, which blocks are enabled, whether anonymous comments are allowed?
Performance/scalability is a major concern to Drupal devs and if you take the proper precautions a site should work well when being slashdotted (e.g. enabling the throttle for resource intensive modules/blocks).
I have had to upload a static page as Markus suggested. Unfortunately the way Drupal works it also invovled uploading a bunch of other directories with PHP scripts that redirect to “node” so the remaining functionality still works. Hopefully that will appease my web hosting lords and masters at £$%^host.xyz, grrr.
WP-Cache 2 is pretty much essential when you host WordPress blog on a shared host. From my own experience WP is pretty CPU intensive, especially with large number of plugins loaded. Page-level cache should indeed be built into WordPress.
As of Drupal, Dries Buytaert has recently posted a comparison between Drupal and Joomla default installation performance, and has shown 500+% gain when cache is turned on. Since caching will only affect guests and dynamic content is still served to logged in users, maybe it is just a good idea to always leave caching on…
A final suggestion would be: don’t host at a place where your domain can potentially be held for ransom. I’ve read stories elsewhere that some hosts will just shutdown the account and deny any access to retrieve the files, unless a fee is paid to upgrade the hosting plan. People looking at web hosting for their blogs should definitely do some research, read through the TOS before committing to a contract.
Drupal does include caching, obviously not enough to save my blog
2 weeks ago my main WordPress blog got BoingBoinged then Farked. Same thing happened to me – over 20% resources, site was essentially down. Luckily I found a WP plugin that does caching and it fixed all of my problems. I wrote about the whole experience in my other blog. I’m going to give the link to the article about it here since I think it might be helpful and it is highly relevant. Hopefully this won’t be consdidered link-dropping.
Brian
Good tips Markus, just a shame I only found out when it was too late and the plug had been pulled
Another two points:
Advanced:
You can also try to make that single page a static one if you start seeing the traffic tsunami rolling towards a particular page. How? Create the directories which show up in your clean URL and publish a static page index.html page in the last directory. Transform your article link into a directory too! Then take the dynamic article offline. In most cases this will work and the CPU load resulting from database pressure drops to zero. To get a server like Apache into a situation where the delivery of static pages becomes critical is really hard to achieve even in a shared environment.
Host big pictures outside of your traffic restricted hosting plan.
You’re definitely right with that list.
Everyone should follow it. It’s not even a trick or something like that : it’s just common sense.
Meanwhile, I do think your host should have contacted you before “cutting” the line. A good time for change
Comments are closed.