Summary:
1. Start simple with technology already there, facing re-scalability issues later OR
2. Write a (to your mind) scalable system that will cost hours of work, eventually performing computationally worse in the very large scale..


Hey guys,

I just started a new project, and this time I am not sure where it will take me from a computational point of view. And I am breaking my nuts and bolts thinking about the issue of scalability.

I know there is a lot of literature out there on the issue. But it didn't really help me on coming to grips when regarding scalability issues.
What I came across, however, are descriptions of how other companies facing the same problems have had. In that sense most started with no abstraction of their software model from the tools they used. Starting with MySQL and Apache was always a good start, it seemed, and later parts of the software have been rewritten.

I mean there are basically the following choices:
1. start with a simple achitecture and code as usual, using mySQL and regular apache, working with PHP, JS, and python.

2. Write own database concept and own webserver that would do what I want them to do. (bogus..)

3. Use what is already there (as in 1) but virtualize the actual architecture from its environment (Webserver = Apache + other stuff), Database (virtual data-space mapped into SQL/XML etc DB spaces), etc.

Forgetting 2 there are two options left. Start simple and see where it goes, eventually having to rewrite your entire code and break your head about how to re-scale the system once people are using your software (yes, it's going to be some sort of social network.) ; write a perfectly consistent and scalable system from scratch and make it work on paper, and spend millions of hours about how to make it work (writing you virtual machines).

Or maybe a trade-off between option 1 and two, using common virtualization methods, which would make the system more scalable, but which could turn out to not give me the consistency during expansion later.

I mean writing software concepts when you know what your program shall do is fun, and I am sure it makes fun, too, to write the VMs. But on the other side, there is no way to tell that there is at all a way to virtualize the system, and even more importantly, the entire system might perform terribly bad after all from a computational point of view. (Desiging the consistent architecture to be comptuationally efficient is one thing, including the VMs and the underlying architectures seems impossible to me given by background).

So what I ask you is what your gut-feeling, what you experience tells you what I should do.

Thanks in advance for any help on the issue. Some ressources would help or maybe there is like this big "tag" on the subject that I didn't yet come across and that I just have to type into google to find all the answers. I didn't manage to do so, yet. So sorry if my question is either trivial or has already been answered.

Sincerely,

Jerrey