Software Design and Construction | Project management | Forum

Hello everyone,

while reading the CSTART website, wiki and software source, I noticed that although you have a software wishlist, there's no information available on how to actually implement the desired functionality. Good programmers don't necessarily know much about physics. As I can tell from the source of usofs.py, there is a clear need for some kind of design principles and a philosophy of communication between experts in different areas.

Here are my suggestions:

1. Think about software design principles
2. Write libraries instead of programs
3. Make engineers/physicists write specifications of the desired software

Here's why:

1. Software design principles enable programmers to assure the maintainability of the code. Ten years from now, e.g. Python might be a completely different language (or might have died), and programmers then might not be able to understand what we're doing today. A general set of rules such as "don't write functions without at least a single line comment describing their purpose" or "use meaningful descriptors" greatly improves the overall code quality. Point in case: what is the purpose of the function rho() in usofs.py? Can you tell without looking at the source? Without asking the author? Without understanding physics?
2. The naive approach of writing large software systems is to start writing applications, realizing that they share common subtasks, and extract those components, collecting them in one or more libraries. You'll need both, programs and libraries. It's easier to start with the libraries. "If what you need is a car, you don't start by building a car."
3. By having a specification of what the software is supposed to do, it is easy for non-physicists to actually write the software. Prepare papers with formulas and explanations, such that ordinary programmers can easily learn what they're supposed to do. It's better to have the software built by experienced programmers who have heard about physics, than by experienced physicists who have heard about programming. Also, comparing specification and code enables programmers to verify and even prove that the software works correctly. The current approach might work for software systems with, say, 10.000 lines of code. It will fail for 2 million lines of code.

I'd like to end with an excerpt from usofs.py, which neatly shows the relevant deficiencies in the areas of design principles, code documentation and user interface design:

def usage():

        usagestr = """Foo""/p>

        print usagestr

Greets,

Philip

PS: if you are the author of usofs.py, this is by no means meant to be an offense. It's just that there isn't yet any other source to take examples from.

Hi Luke,

software engineering is indeed something I'm enthusiastic about, and although I consider myself to be somewhat knowledgable in this area, I certainly lack the years of experience necessary for making informed decisions. I do know about all the major software design models with their respective pros and cons, and I know a fair bit about the process of developing software, i.e. coding, testing, debugging, profiling, optimizing, distributed environments. I'm an acceptable programmer (in descending order: C, Python, SQL, C#, C++, Perl, Pascal, Go, Brainf*ck, Scheme, SML, …), though I prefer thinking about algorithms rather than writing specific code, although I consider myself to be a good reviewer.

So this is me. As you may have already guessed, I'm pretty bad at physics and English. Anyway, I'd be glad if I can be of any help, though I don't know how much time I will be able to devote to CSTART.

Do you already have specific questions regarding software engineering processes, design models, or code development in general?

Hi antinode,

the waterfall model is what first came to my mind when I wrote my initial posting. I didn't want to talk about something as important as the overall design strategy, but now I'm glad you asked.

Before I'm going deeper into the issue, I'd like to mention that it's generally a bad idea to adhere to a certain software development methodology only because it has proven to be acceptable in hardware development environment. I once had to build a simple (but complete) CPU from scratch, and the waterfall model surely was the best choice, but it also has huge disadvantages under certain circumstances. That being said, the waterfall model is still a good choice in this particular case, although I will propose something inherently different, yet deeply related to the waterfall model.

So, why is the waterfall model a good choice? "Big Design Up Front" design strategies make the assumption that software development can be split into subtasks such as requirements phase, design phase, implementation phase and so on, and that each phase can be completely finished before any work on the next phase has begun. This assumption is usually wrong, but in the case of CSTART, this might be the case: we don't build a software that nobody has ever seen yet, we don't have a customer that adds new requirements 3 months before the final release, and quite a bit of the stuff we will be implementing is pure math/physics anyway. Furthermore, the waterfall model ensures that their will be documentation available, which is especially valuable in an environment where developers leave and join on a frequent basis. And of course, the waterfall model is easy to understand. If anyone asks "where are we right now?", he can be answered by e.g. "the end of the design phase, here are the docs". Compared to, say, Extreme Programming, this is a huge advantage.

Their are some downsides, too. The waterfall model requires a very high level of discipline among designers and developers. It takes months, in our case years, before the first line of code is written – and this makes it boring for most programmers. Who wants to talk and talk with physicists/engineers over the course of years without ever producing something that… works. In the short run, there's no payoff. I doubt that we will be able to keep designers and programmers motivated throughout the duration of the whole project. In my opinion, this is the main disadvantage. In the worst (yet still highly probable) case, the design people will have left the project before the programmers get to implement the software. Now, imagine what happens if there occurs an error during the final mission. Who's to blame? "Oh, that's Steve's fault. He left the project about two years ago." Furthermore, with the waterfall model, every individual who leaves the project before it's finished takes highly valuable knowledge with him that can't be replaced without some guy reading the docs for half a year.

So. In my opinion, the waterfall model would be a good idea if we had $10.000.000 at our disposal to keep people motivated. It's a bad idea for teams with a member fluctuation greater than zero.

Hence my proposal is an agile software development approach: we split up "the software" into modules that are as small as possible, and use a slightly modified waterfall model to solve all these sub-problems. Each module is maintained by its own small team, which is responsible for everything that has to do with the module, including all the different design phases. If the final outcome doesn't satisfy our needs, we iterate. Besides, we don't use the traditional waterfall model, but the Sashimi model, where phases are allowed to overlap.

Features:

• one iteration of the waterfall model for a sub-problem probably takes 4 weeks instead of 4 years
• hence high motivation by defining goals that can be accomplished in a
  reasonable amount of time
• still the same benefits: lots of documentation, unit tests, validation, …
• small teams that interact with each other, not one large team that drowns in chaos
• easy to maintain high disicpline among team members
• teams are able to respond to changing requirements / new insights quickly
• we can learn from mistakes without throwing away everything we did thus far
• small, independent teams
• maybe even some kind of competition among different teams (fewest bugs, shortest implementation time, highest efficiency)
• easy to maintain high communication volumes among team members
• and many more

This approach is, in my opinion, most suitable for an environment like this, without sacrificing the substantial benefits of the traditional waterfall model. I might have forgotten something, but I'm eager to discuss the issue in more detail anyway.

What are your thoughts about this?

Hi Luke,

if you're interested in learning about software engineering and particularly design philosophies, I recommend reading about the waterfall model (which is often referred to as the "traditional" design methodology) and then about Extreme Programming (which is considered to be the other end of the spectrum). While reading about those two, you may eventually find useful pointers to other design strategies. Agile software development, as proposed by me, bridges the gap between the traditionalist approach and the radical newer ones. The Wikipedia articles are a good starting point, and I may have some scripts from lectures about the topic in general. The C2 wiki (whose founder is also the inventor of wikis in general)
used to be a well of information about design strategies (and, mainly, design patterns).

By the way, I've been reading the stuff on your website, and noticed that you are in the process of learning German. Incidentally, I'm a German native speaker, so if you need someone to further your abilities, I'd be happy to help out. I'm from Saarbrücken and found your site by googling for Dennis, so maybe this is not so incidental.

Regarding your question whether CSTART needs different design processes for different parts of the software, I do not know. Maybe there are completely non-critical parts where reliability is not a concern, but that doesn't affect the choice of the design strategy. Reliability is a main concern of every (sane) design strategy available (see "Cowboy coding" for counter-examples).

Maybe I will have to make something clear: the design strategy doesn't necessarily influence the way the final product looks like. Given a large piece of source code, you probably can't decide whether it has been built using waterfall, XP, agile, whatever. The design strategy merely defines how to get from the idea to the final software, and these processes may look and feel entirely different.

Furthermore, the agile software development approach is not "heavy-weight". It's not "light-weight" either. It's a heavy-weight process made light-weight. Maybe one could call it "middle-weight" or better "adjustable-weight". In order to achieve a higher software quality, the whole process is simply repeated, thereby adjusting the previous outcome (and increasing the weight).

This has some advantages. First of all, if the outcome doesn't look satisfying, you simply iterate. But in contrast to other iterating design strategies, you'll have documentation available right from the beginning. You can omit writing unit tests or having code reviews, but if it later turns out to be useful, you start another iteration and add those. It is completely adjustable; for a small project like the scripts you wrote, it would suffice to come up with a preliminary manual and then write down the code, adjusting the manual as needed. For larger projects, you add whatever you need.

The problem right now is that with a more light-weight approach, we don't get documentation until someone reads the code and writes some manual. The agile approach, in its simplest form, merely reverses the order. Write some docs first, then write the code. Iterate as needed. This way, the software can "grow" in a controlled manner, documentation and code both keep up-to-date, and more advanced techniques can be added easily as needed.

I wouldn't recommend two different design strategies, but merely several levels of "how accurately do we want to mimic the look and feel of waterfall per iteration". Small projects may be done in one iteration (and one day), larger projects may need 50 iterations and two years.

By the way, what exactly are we talking about when we're referring to "the software"? The whole collection of applications and libraries related to CSTART? Everything the users will eventually come to see and work with? Everything that is not "the hardware"/p>

And I have another question: which operating system will run on the spacecraft? Because thus far, I think all we have been talking about is applications and maybe libraries. But it may be important to know something about the operating system before any serious application design can start. Do you intend to take an existing operating system? Modify it to suit your needs? Or build it on your own completely from scratch? Will there be a difference between the operating system and the applications that run on top of it? Is it a multitasking operating system?