The Daly Premise
by Timothy Daly Sr.[@]
I have resisted the blogging trend for many years, spending all of my
nights and weekends on Axiom. Continuing that trend, this blog will
be about Axiom related thoughts and opinions.
Does Axiom have Quality?
I have, for many years, been wrestling with the question of quality.
If you have read "Zen and the Art of Motorcycle Maintenance" then you
have some idea of how deeply the issue goes. If you haven't read it then
this post will be nonsense.
What does quality mean in terms of a project like Axiom? According to
Persig you can know if Axiom has quality without needing any objective
criteria to judge. I agree with this sentiment.
However, as the lead developer of Axiom I have to reduce the quality
question to practice. What, exactly, are the things that must be done
to ensure that "Axiom has quality".
At the moment I would say that Axiom is not a quality product. It
does follow the state of the art in a lot of ways but the "state of
the art" is sorely lacking. In fact, I can see that there are areas
where I had not previously considered even the very beginnings of
the question if you look at Axiom from the perspective of a user.
This was brought to my attention with an article by Patrick McKenzie
titled "How To Successfully Compete With Open Source Software"
[1]
Patrick raises 6 areas that allows him to compete with open source
software despite the fact that he has no real technical advantages.
1) Marketing
2) Design
3) User Experience
4) Speaking the users' language
5) Support
6) Technical superiority
I won't summarize the details here but he does provide a good basis
for judging Axiom with a "users' eye". In all of these areas Axiom is
failing badly. It lacks quality from a users perspective.
Axiom, like a lot of other software projects, is based on a style of
technical development that strongly emphasizes the developer's
viewpoint. This viewpoint has several components.
1) New features
There is a strong concentration on introducing new features. In a
computer algebra system like Axiom this usually means new algebra.
Since the ultimate goal of Axiom IS algebra, it gives the development
of new algebra that "shiny bit of foil" attraction.
2) Interest focus
Tied to the prior point, the new algebra is strongly related to the
current interest of the developer set. Thus, the algebra that is deemed
important or that gets developed depends on the current developers.
The upside of this is that the developers are "scratching their itch",
which has been claimed as the primary reason why anyone does open source.
Thus, the developer is likely to spend a considerable amount of effort on
obscure details in order to settle specific questions.
The downsides of this are numerous. First, the newly developed code is
likely to work for the particular example given by the developer. It is
unlikely to work for anyone else because it makes assumptions that are
unknown and not generally valid. Second, because the developer is an
expert in the area there is an assumption that everyone who uses the
code is also an expert. Third, in a system as complex as Axiom, there
are correct and incorrect ways to add code to the system. It takes a
long time to understand the conventions (which are nowhere documented)
and standards (again, not documented) for new code.
There are systemic issues that are generally ignored because they are
tedious. There is no documentation of the details of the algorithms.
There is no documentation for the user. The global documentation of
the system is never updated which means that even the documented
functionality slowly rots away. There are no test cases and no
regression tests. There are no boundary cases tested. In short, the
usual case is just to add a new bit of code to the "code pile" and
hope for the best.
3) Bad standards
Open source software has developed a set of standards over the years
that have grown out of common practice and solve a number of technical
issues.
4) Technical focus
Given that Axiom is in such a limited domain, the intersection of
computer science and mathematics, most of the users are other experts.
However, as Patrick points out, they may be expert mathematicians but
they are still users. They have problems in math that are not really
perceived as computer problems. They expect a high quality and useful
interface as opposed to a command line. They expect strong support for
developing their problem statement and its solution. They expect a
vocabulary with matches their own. They expect support. And they
expect technical superiority.
The current "gold standard" in the computer algebra area is Mathematica.
It provides all of these things in some measure or other. There is an
expectation of high quality (which is a point of debate among experts
as to whether Mathematica (MMA) is of high quality technically). But
Mathematica gives a very pleasant notebook interface which allows the
user to develop their problem statement along with their solution. It
allows the user to develop their own vocabulary. It provides support.
Axiom has been lacking in all of these areas except in technical
excellence. Axiom has a much stronger model for its algebra hierarchy.
But the user has many hurdles to cross in order to appreciate that fact.
That said, there has been quite a bit of effort expended to seriously
improve Axiom's quality. I will cover some of these in the next post.