Build a Twitter Analytics App
2 The First Step: Design Your Solution
6 Writing the Backend Twitter Server
Writing the Code in Small Parts: Part 1, The Basic App
Part 2: Adding a Counter to Exit
Part 3: Adding Language and Retweet Count
7 Adding the Data to a Database
8 Testing: What and How to Test
9 Displaying our Data using the Flask Webserver
9.2 Adding templates to our Flask app
9.3 Displaying our Tweets in the Flask Web Server
10 Future Work and Improvements
On a public forum, I got a rather snarky comment saying there wasn’t a single line of code in my tutorial, and I was wasting people’s time. Even though the person who said this is clearly a troll, I thought I’d explain a bit why we are building this project this way.
I have been working as a software engineer for years now, and I guess things which maybe common sense to me, are not common sense to a new programmer. So let me tell you this now. Some of you maybe shocked.
If you go work for any of the top software companies, only about 30% of your time will be spent coding. For an average project, you will spend:
- 30-40% of your time designing the solution. Yes, this much. The more time you spend early on, the less you need to spend fixing bugs later.
-
Another 30-40% writing the code, which includes writing tests.
-
10-20% debugging and testing.
-
10% in code reviews.
I realise it adds up to > 100%, but I’m just using rough figures. And the above doesn’t include time in meetings, or other busy work.
That means that only about 30% of your time is spent coding. So if you complete something like Codecademy, you only know less than 30% of what you need to know. Actually, a lot less, as real programming includes knowing things like writing clean code, organising your code, writing code that can be maintained for 5+ years.
Here’s the thing: Most companies won’t tell you this. They will just expect you to know it. But if you turn up at a good company and start writing code without first thinking about the problem, you will be told off.
Software Engineering is an art. Being proud of your work, doing work like an artisan is what separates us from code monkeys.
Say you hired someone to build our home. The builder turns up, his pants hanging down to his ass, cheap rolled up cigarette hanging from his mouth, and says “Yup, we’ll build your house in a week.”
And he starts throwing bricks willy nilly, in the shape of a wall. But a few days later, he realises he is going at a 30% angle, so throws everything down and starts again. A few days later, he realises he built a wall where there should have been a window. The builder laughs and says, “Mistakes happen. What are you gonna do?”
What will you do? Will you ask him why he didn’t design blueprints of the house before he started throwing bricks around?
“Hey, I’m a builder, not a designer,” he says.
Do you see now how many business owners see programmers? Just like that builder, pants hanging down, boasting, “Yeah, we’ll build that website in a week. No problem.”
Most of the problems we faced in programming today have been faced before, and thought about. The Mythical Man Month was written in freakin’ 1975, and talks about many of the problems a programmer in 2015 would face.
Many programmer complain they are treated with respect by management. Well, act like a professional. Be proud of your craft.
For most software companies, writing code is not the problem. The biggest problem any software company will face is managing complexity. You will have dozens, if not hundreds, of programmers working in parallel on the same code base, adding features, fixing bugs, refactoring code to make it more maintainable. Code that was written five years ago might suddenly develop a new bug when it is used in a new place. It is complete chaos.
The only way to manage this chaos is to have the highest standards. Most of the time, this just means communicating what you are doing. Communication is the key thing which separates the senior/architect level people from the beginners. Most failures in software companies, including the big blowups you read about in the papers, happen because of poor communication. Someone assumes something will happen in a certain way / on a certain date, and never bothers to check.
And that’s why everything must be done by a process. Process becomes irritating when it becomes an end in itself, but having no process or poor processes is worse than having too much process. And in this case, the process we are talking about (design-code-test-repeat) is led by engineers, for the engineers, so there is no reason not to follow it.
And now, on the orders of the boss (my wife), I will stop my rant.
I will say this though: These things are easy to learn, provided you are willing to learn.