Accumulate evidence

Wednesday, October 26, 2011

It's not a dutch project, is it?

A few months ago, I had done some work as a contractor for a client that became a victim of its own success.  The company thought its prime objective was to make profit, but after about an hour of analysis, the actual purpose of the company the total acquisition of as much technical debt as possible.  After struggling with the setup for months, I had decided to confer with fellow professionals about alternative approaches:


<12dCode> you know you've hit the big league when each PHP instance allocates 512MB peak 
RAM <12dCode> on production <12dCode> for your main product <GoogleGuy> 12dCode, You mean you're not using mod_php :P <Remi_Woler> the big leage of people who don't know how to spell optimization? <12dCode> i'm not even accounting for the apache spawn <GoogleGuy> 12dCode, pmap should be telling you a very different story then <12dCode> dunno what that is <12dCode> but i do know that memory_get_usage() is saying it's allocating half a gig per
instance <Remi_Woler> shame. It should say "you're doing something really really wrong" <12dCode> lol <12dCode> oo i see it now <GoogleGuy> 12dCode, http://linux.die.net/man/1/pmap <Phergie> [ http://is.gd/ailJrI ] pmap(1): report memory map of process - ... <GoogleGuy> I don't know what you're doing that needs 512MB of memory in PHP, but I sure
hope you have enough memory to scale that way <GoogleGuy> heh <Remi_Woler> replace the ORM with a handful of queries: back to 12M :P <GoogleGuy> Remi_Woler, You mean take out the stupid 50,000 lines of code that people
think is better and easier than writing a few hundred lines of SQL? <GoogleGuy> nah, that can't be right <waerloga> GoogleGuy: of course it isn't right, it's not OOP and webscale! <waerloga> :P <Remi_Woler> if it even gets to a few hundred. Usually it's a dozen, maybe even a couple
dozen, one or twoliners <12dCode> we have 5 layers of cache <GoogleGuy> Use MongoDB and become the next twitter <GoogleGuy> You will have a fail whale on your site to replace the front page <GoogleGuy> Yes, as you can see Ruby and MongoDB scale very very well <Remi_Woler> GoogleGuy: I believe scaling only counts if it goes up horizontically or
vertically <Remi_Woler> s/or/and\/or/ <12dCode> table queries are pregenerated into php objects and put into files. those
files are then aggressively loaded into APC. if the record is there, we load it from
APC and into a cache in the local instance of PHP. but if the "record" isn't in APC,
we try to pull the file directly. When that fails, we try to pull it from the db <12dCode> it should be noted our langauge system is comprised of 30 massive tables <GoogleGuy> 12dCode, That sounds like reinventing an rdbms <waerloga> viva cache misses <GoogleGuy> albeit reinventing it horribly and with far worse performance <12dCode> correct <12dCode> all sorting and queries take place in the application layer <GoogleGuy> heh <waerloga> ew <GoogleGuy> how's that working out for ya <12dCode> because PHP is really good at that <GoogleGuy> o.0 <12dCode> having no pointers and all <12dCode> who needs a b-tree? <GoogleGuy> You just mentioned the one thing PHP sucks at <12dCode> pfft <12dCode> linear search solves everything * 12dCode cannot sarcast enough. <GoogleGuy> I couldn't tell for a moment honestly <GoogleGuy> You sounded pretty serious their for a moment <Remi_Woler> 12dCode: it's not a dutch project, is it? <12dCode> anywho, we can concurrently serve 6 users per machine <GoogleGuy> 12dCode, That sounds like the next Google <GoogleGuy> or the Altavista <GoogleGuy> I can't tell <GoogleGuy> One of them was successful supposedly <12dCode> when we try to implement solr or mongodb to act as a querable in-memory
caching solution, we get told to continue to add shit to our shit pile <Remi_Woler> 12dCode: hey, at least the piles are sorted

No comments:

Post a Comment