SOLID Principles and the Arts of Finding the Beach

If you look at the picture below I’m pretty sure you would find the beach following the sign would you?

beach-sign

Now imagine you wouldn’t know what a beach is, what would you do? Chances are that you miss the beach, swim around in the sea and finally write an angry blog post or tweet bashing the sign as totally misleading [5] and the person who put the sign here in the first place learns a invaluable lesson:

It is really hard to give guidance that is understood by everyone as it was originally intended.

Or modern world is running on computers and computers are fed with power and software so there is an ever increasing demand of skilled software developers to tell the computers what to do and when and how. Those software developers are coming from different countries with cultures, different education systems and experience so it’s increasingly hard to receive or give guidance without misunderstanding or being misunderstood.

Logic and Mathematics

Fortunately, we have methods in place that are thought in schools, colleges and universities all over the world, so they can be considered as commonly understood. They are called logic and mathematics [1]. The basic idea behind both can be described as follows:

  1. Set up general, arbitrary rules (also called assumptions or axioms).
  2. Draw conclusions that meet the given rules.
  3. Show unambiguously and in a way that is repeatable by anyone that your conclusions are consistent with the rules. This demonstration is called proof.

This method also called deduction might remind you of a game and this isn’t a coincidence. Every game has rules and every draw has to be consistent with the rules. Those rules are the same for every player and also accessible by anyone who is interested. But unlike most games, mathematics and logic aren’t based on competition, everyone can contribute and find a new conclusion and everybody who finds a new proof is a winner.

Science

No doubt, the deductive method used in logic and mathematics is incredibly successful but it doesn’t tell you anything about the real world. This is because we don’t know what the rules of the world or the universe we live in actually are, nor do we know if such rules actually exist. So all we can do is observe the world, identify patterns and unify them into scientific laws or theories.  The method of gathering information and identifying patterns is called induction (not to be confuse with mathematical induction which is actually a form of deduction). It is also immensely successful but it has an obvious problem: We cannot observer everything that’s going on in the universe, hence our information thereof is always limited. So every scientific theory is only valid as long as nobody finds a counter example that renders the theory invalid or invalid under certain conditions.

Karl Popper illustrated the fundamental flaw of the inductive method with the following thought experiment [2]: Suppose you observe swans all your life and all you see is white swans, does this prove that all swans are black? No! You have no evidence that you have seen all swans and hence you cannot tell if there isn’t a swan on the planet that isn’t white.

Despite of this problem, science has served us pretty well. One reason for this is that it demands the same openness and transparency that logic and mathematics demand: Every theory can be verified or falsified by everyone else independent of the inventor of the theory. So over the years, generations of scientists had the opportunity to test and retest central scientific theories over and over again.

The Engineering Method

All fine but what does this tell us about software development or better software engineering? Engineering is all about building stuff and engineers use the theories of science as well as the methods of logic and mathematics but they are confronted with yet another problem: There is usually more than one way you can build something to implement a given functionality.

Fortunately, we have many constraints in traditional engineering. The vast majority of possible implementations would be too big, to heavy, consume too much energy, wouldn’t be possible to build or would be way too expensive. This is often very frustrating but at least we can find out what works and what not.

Due to the incredible flexibility of computers, the set of possible solutions is much larger in software engineering than in any other engineering discipline.

To find the right direction in the maze of possible solutions, engineers have developed “hint signs” named heuristics, rules of thump , principles or laws [3]. They aren’t faintly as strict as rules or conclusions in mathematics nor as well tested as scientific theories, just useful little helpers that keep us from doing something we might regret later.

Back on the Beach

And here we are back on the beach. The sign isn’t meant to be a tool that defines precisely the location and geometry of the beach. It assumes that you know that you are on a beach when you are on a beach. It points in a direction and assumes that you know where to stop when you are there.

The same is true for the Principles of Object-Oriented Design aka SOLID Principles [4]. You need some experience with object-oriented software development in real world projects before they are of any use to you. Furthermore, they don’t tell you when you have found a good solution (the beach) and when you went too far (into the ocean). Going too far means that you are outside the “scope” of a principle and the expected advantages don’t appear.

Conclusion

Principles, Heuristics, ‘laws of engineering’ are like hint signs, they are helpful when you know where they are pointing to and you know when you have gone too far. Applying them requires experience, that is trying things out, failing, analysing, talking to people, failing again, fixing, learning and failing some more. There is no short cut as far as I know but if you find one, please leave me a comment!

Further Reading

[1] Bill Shilito:Introduction to Higher Mathematics, 2013, https://youtu.be/CMWFmjlB8v0

[2] Wikiopedia on Falsifiability: https://en.wikipedia.org/wiki/Falsifiability

[3] Billy Vaughn Koen: Discussion of the Method, 2003, https://global.oup.com/ushe/product/discussion-of-the-method-9780195155990

[4] Robert C. Martin: The Principles of Object-Oriented Design, 2005, http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod

[5] Dan North: Why Every Element of SOLID is Wrong, 2017, https://speakerdeck.com/tastapod/why-every-element-of-solid-is-wrong

[6] Charles Petzold: The Annotated Turing, 2008, http://www.theannotatedturing.com/

[7] Gerald M. Weinberg: The Secrets of Consulting: A Guide to Giving and Getting Advice Successfully, 1986, http://www.geraldmweinberg.com/Site/Consulting_Secrets.html

[8] Donald C. Gause, Gerald M. Weinberg: Are Your Lights On? How to Figure Out What the Problem Really Is, 1990, http://www.geraldmweinberg.com/Site/AYLO.html

The meaning of “Estimate” in #NoEstimates

Proponents of the #NoEstimates “movement” don’t use term Estimate in a way that is consistent with common English an Estimation Theory but their definition of Estimate is essential to understand what the movement is already up against.

Henceforth, I write Estimate in bold and upper case when I am referring to the #NoEstimates jargon.

An Estimate is the answer (number) you get when you ask a software developer or any other person how long it will take to complete a certain piece of work. The number can have a time unit such as hours or days, or it can be a relative metric such as Story Points. Apart from a number it can also be a probability distribution in any of the units mentioned before.

Predictions and forecasts based on historic data are not considered Estimates. Furthermore, approaches where a certain time budget is reserved for a piece of work is also not considered an Estimate.

As always, corrections and clarifications and constructive feedback are welcome.

OWASP Top 10 Relaunch

After all the disputes around the OWASP Top Ten 2017, the project leaders have decided to make an “Update” to the whole project which could be interpreted as a relaunch. Here is a summary of the most important changes:

  • New team of OWASP Top Ten leaders.
  • Scoring for Top 10 entries is intended to be based on Common Weakness
    Scoring System (CWSS) as opposed to votes from selected OWASP sponsors.
  • 8 of 10 vulnerabilities will be selected from
    data submitted via the call for data. The call for data was therefore reopened.
  • 2 of 10 will be selected from an open survey.
  • The schedule was updated as follows:
    Deadline Date
    Survey 30 August, 2017
    Call for data 18 September, 2017
    The Top 10 2017 RC2 release 9 October, 2017
    Final release 18 November, 2017

I really appreciate the team’s decision to make a clean break and their commitment to transparency and focus on actual software vulnerabilities and keep out organizational flaws such as “insufficient attack protection“.

That doesn’t mean however that organisational and architectural flaws such as “insufficient attack protection” or “lack of an overall security strategy” should not be addressed by the OWASP! I am convinced that some guidance in this regard would be highly appreciated in the industry.

Personally, I am not a huge fan of surveys because of all the pitfalls involved in interpreting the data.  However, all the data is accessible to the public so we can draw our own conclusions.

Furthermore, there is a commitment to make it easier for people who are no security experts to understand the problem and to provide guidance on how to fix it:

Every single issue in the OWASP Top 10 should have a direct cause: either one or more missing or ineffective controls, or not using an in place control.

Every issue should be precise in its language and view (as in not intermingling the terms “weakness,” “vulnerability,” “threat,” “risk,” or “defect”) so each issue can be theoretically testable. […]

[…] There should be a view from the Developer perspective (documented by the OWASP Proactive Controls) and a view for the Defending Blue Team (documented by the currently non-existent OWASP Defensive Controls).

Every issue should contain clear and effective advice on remediation, deterrence, delay and detection that can be adopted by any development team – no matter how small or how large […].

Conclusion

The openness of the data as well as the process of the OWASP Top Ten gives us the opportunity to either use the new Top Ten “as is” or use the data to create our own list of vulnerabilities. A list tuned to the specific needs of our organization our our clients as I have shown in The Real OWASP Top Ten. Now we get more data, more transparency and better guidance. We couldn’t ask for more, could we?

The Real OWASP Top 10

The OWASP Top 10 is list of the top ten most critical web application security risks published by the OWASP. Since its first release in 2004, got a lot of attention and was referenced by several security standards such as the PCI DSS, the credit card industry security standard. The Top 10 are published in a three year cycle that ended in 2016 due to internal disputes in the organistaion.

Other standards such as the PCI DSS didn’t follow the OWASP Top 10 release cycle of the OWASP but forked of their own list of security risks in web applications. As a result, the current struggles to come up with a new release of the Top 10 doesn’t have the impact on the security community as many of the project members believe. However, the Top 10 were always a good starting point for people to get into web application security.

Unfortunately, some of the risk expressed in the recent release candidate are unspecific, redundant and lack precision. “Insufficient Attack Protection” for example includes protecting against every web application vulnerability expressed in all the other Top 10 topics as well the ones that haven’t made it into the Top 10. The same is true for “Unprotected APIs” as Web APIs can have the same vulnerabilities as the rest of the web application and “Sensitive Data Exposure” which can mean anything.

One the plus side, the OWASP is very transparent and the data on which the current release candidate is based is hosted on github. After applying some grouping, I got the following result:

OWASP Top 23

Now compare this to the official OWASP Top 10 2017 Release candidate:

OWASP Top 10 2017

It is interesting to see that neither “Insufficient Attack Protection” nor “Unprotected APIs” seems to be based on real observation whereas  Path Traversal with a frequency of almost 15% (top 3 excluded) hasn’t made it into the OWASP Top 10. Furthermore, Mass Assignment vulnerabilities which are as dangerous as SQL Injection and far more frequent than CSRF have been dropped, yet OWASP chose CSRF. Maybe this is because it is much easier for web application firewalls to prevent CSRF than Mass Assignment.

Conclusion

Concentrating on the OWASP Top 10 2017 alone is not sufficient as highly dangerous and easily explorable vulnerabilities such as Path Traversal and Mass Assignment have been sweeped under the carped of politics and tool vendor interests. So maybe OWASP should trash the Top 10 altogether and come up with a more comprehensive list of risks. A list that people can use as a solid starting point for digging into web application security.

Added 08/22/2017: There is a followup OWASP Top 10 Relaunch.

Estimating Fixed Price Projects in Open Source Teams

Some time ago, I have demonstrated that it is well possible to plan fixed price projects without direct time estimates despite the fact that the team velocity is permanently changing. Now I was wondering if the method could be applied to open source projects too. Unlike traditional software teams, open source teams are permanently changing and they usually don’t estimate effort estimations but nonetheless produce world class products.

So I picked some projects from the Apache Foundation, namely some of those who use Jira so I could easily extract their work history and do some analysis on it. The first challenge was to extract the velocity from the given data as the teams didn’t estimate in story points. So I looked at the issue types used in those projects and their cycle time histograms. Here is an example from the Apache Zeppelin Project:ZEPPELIN_category_cycle_time.pngI only considered issue types that were not part of other issues so the only ones left were “Bug”, “New Feature”, “Improvement” and “Wish”. From the cycle time histogram, we can see that the average cycle time of Wishes and New Features are roughly two times the average cycle times of Bugs and Improvements. So I assigned each Wish and New Feature two story points and one for each Bug and Improvement. After introducing 14 day sprints I got the following velocity distribution and average cycle time per sprint:

ZEPPELIN_velocity_cycle_time_history.png

A fictional project

In order to do some estimation and check its accuracy, I picked an arbitrary sprint in the middle of the graph and considered only the velocity up to this point (in our case Sprint 1 to 40) for further observation. The result was the following velocity histogram:

ZEPPELIN_prior_velocity

Now I chose to estimate a 200 story point project (bug fixes included). How long would it take the team to finish?

The maximum velocity the team had ever reached was 45 story points so the minimum number of sprints to complete a 200 story points projects would obviously be 5 sprints but with a probability of only 2.5%. The average velocity was 17.93 story points, hence the average number of sprints to complete the project would be 12 sprints. The 95% case requires a Monte Carlo simulation. After 5000 simulated projects, I got the following distribution:

ZEPPELIN_simulation_result_distribution.png

So if we would have committed to finishing the project within 16 sprints, we would be successful in 95% of all cases. Now lets see what really happened after sprint 40 (the blue line in the graph below):

ZEPPELIN_velocity_total.pngAs the team continued to increase velocity, they managed to finish the fictional project in Sprint 48. It only took them 8 sprints to complete 200 story points which was even better than the average of 12 sprints and twice as good as the pessimistic 95% estimate!

Estimating the profit is less straightforward in open source projects than with stable teams (teams that don’t add or remove developers during the project) as the fictional cost per sprint changes with the number of contributors. We’ll have a look at this in the next post.

#NoEstimates needs Disambiguation

As Henrik Ebbeskog points out in his post Why not estimating doesn’t make sense, Woody Zuill’s definition of the hashtag #NoEstimates doesn’t make sense:

#NoEstimates is a hashtag for the topic of exploring alternatives to estimates [of time, effort, cost] for making decisions in software development.  That is, ways to make decisions with “No Estimates”.

It’s actually a contradiction to everything Woody Zuill and others using the hashtag #NoEstimates have ever written. It should better be something like the following:

#NoEstimates is a hashtag for the topic of exploring alternatives to the way we come up and deal with estimates [of time, effort, cost] for making decisions in software development.

The problem is that you cannot really think about the future in a rational way without estimates – nor can you do any planning. So even if you choose to implement the most important features as Woody Zuill Suggests in No Estimate Programming Series – Intro Post, you have to estimate several things. You are estimating which features are important. You have to estimate whether you can implement those features without going bankrupt, so you have to have an idea about the effort and this idea is in fact an estimate.

When you read Vasco Duarte’s book it’s all about estimates. Note that it doesn’t matter if an estimate is binary (can probably do within a certain limit or not), or in story points. Even if you split backlog items to roughly the same size you have to estimate whether the backlog items have the same size. Even if estimates are produced by statistical methods they are still estimates. The only thing that’s different is the way how we should come up and deal with estimates and in my opinion, that’s the thing that should be in the definition.

Added 03/22/2017: I just figured out that the hashtag #NoEstimates is a brilliant example of Paradoxical Intention. In that it is a contradiction and our brains don’t like those. So we ponder about it and the more we ponder the more are we forced to analyse and rethink the way we deal with estimates.

Added 08/26/2017: The meaning of “Estimate” in #NoEstimates.

No Estimates in Practice

On December 11, 2011 Woody Zuill posted the first tweet with the Hashtag #NoEstimates

FirstNoEstimatesTweet

announcing the blog post No Estimate Programming Series – Intro Post in which he describes a fictional project that is using a No Estimates Approach as he calls it. In short, the team implements the most important features as quickly as possible without estimating.

Tracer Bullet

This reminds me of the Tracer Bullet approach described in Andrew Hunt’s and David Thomas’ The Pragmatic Programmer – From Journeyman to Master from 1999. In an interview from 2003 with Bill Venners, the authors describe the method as follows:

Dave Thomas: The software analog to firing heavy artillery by calculating everything up front is saying, “I’m going to specify everything up front, feed that to the coders, and hope what comes out the other end is close to my target.” Instead, the tracer bullet analogy says, “Let’s try and produce something really early on that we can actually give to the user to see how close we will be to the target. As time goes on, we can adjust our aim slightly by seeing where we are in relation to our user’s target.” You’re looking at small iterations, skeleton code, which is non-functional, but enough of an application to show people how it’s going to hang together.

the approach is elaborated in more detail in Jared Richardson and Will Gwaltney Jr’s book Ship It! from 2005.

In contrast to Woody Zuill’s No Estimates Approach, Andrew Hunt and David Thomas recommend detailed estimation but not to make managers happy but to identify critical issues as early as possible. However, this was 1999 when there were no Unit Tests, no Continuous Integration and Deployment etc.

The Lean Startup

A similar approach to Tracer Bullet is described in EricRies’  The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses from 2011. From The Lean Startup on Wikipedia:

A minimum viable product (MVP) is the “version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort”. The goal of an MVP is to test fundamental business hypotheses (or leap-of-faith assumptions) and to help entrepreneurs begin the learning process as quickly as possible. […]

Kanban

In his book Kanban: Successful Evolutionary Change for Your Technology Business from 2010, David J. Anderson argues that from the perspective of Lean, estimation is waste as it doesn’t provide value to the customer. In the project he mentions, backlog items are limited to 15 person days. If a backlog item was estimated to take longer than this, further planning was required. There was no further estimation done for shorter backlog items. Instead, the planning was done using data from prior work. Ralf Westphal shows in his blog post Take a bearing, don’t estimate from December 2016 how this can be done. In essence, he categorizes tasks and records the cycle time for each category. Then, he uses the cycle time distributions and the Monte Carlo method for project planing.

Practical Experience

My personal experience with developing products in small increments predate Lean Startup and Kanban so we used the Tracer Bullet approach. We developed a little bit of functionality with the goal to ship it as soon as possible. Some of those features didn’t even have a UI. We offered the new features and if customers wanted to use it, we improved the features as needed. We also dumped several features – especially those that we implemented just because our competitors offered them and which turned out to be not exactly what the customers wanted. We even dumped our first product line and helped the customers who used those products to migrate to another company.

We did it without estimating, not because we considered it as waste but we didn’t know how to do detailed planning it in an ever changing startup environment. It worked fine for a while. However, as the company became successful, managers demanded project plans and more detailed estimates. This was very frustrating as those activities were a total waste of time but by the time I didn’t know what to offer instead.

I’ve observed a very interesting implementation of Lean Startup the other day. The guys are writing a smart phone app and their key metric is the time users spend using the app and their features. This metric is driving the whole development process. They add features in small increments. They extend the ones that are used as long as it increases the time their users spend using them. They drop those which nobody wants to use after a while. They have a time budget for new features. If it takes to long to build a feature, they try to cut it down or fake it to get feedback sooner.

Conclusion

There are many approaches used in practice that don’t require effort estimates. However, they are not practical in all circumstances. If the customer has procurement rules in place which demand up front specification to compare offers, there is no way to escape the need for estimates – however inaccurate and expensive they are. Even more waste than estimation are the unused features that are produced with big upfront analysis and design. Especially in large system, unused features are all over the place while functionality that is essential for the users is missing or practically unusable. According to Jeff Sutherland about 64% of features are never actually used, so I think it’s time to shift the focus from delivering on time and budget to building the right product. The users would really appreciate it.