Zahid Amin

Founder BackButton browser backbutton Movement

14/11/2019 -

Mission: Master Industrial Strength in Large or Complex Systems

Grady Booch {IBM}: Industrial Strength Software is beyond the intellect of a developer
This is a quote from my HND Computing course recommended text book, Object Orientated Programing Using CPP.

There are two distinct points in this sentence. first point is that there are two types of software Industrial Strength Software and software which is not industrial strength.

Then there is the suggestion Industrial Strength software is beyond the intellect of a developer . But it is the developer who is going to be building the software product, so how is the developer going to do it ? There is a very complex philosophy embedded here.

The book was teaching Object Orientated concepts using C++. C++ is sometimes referred to as C with classes. The intended audience of the book is the application developer and the use of component technology. Industrial Strength is only beyond the intellect if the developer tries this effort on his own. Does it mean Developers should work in teams of two or more ? No it doesn't mean that either. It means there are a chain of developers with each one acting as links. So when developers work as a chain then Industrial Strength Software is within the intellect of the application developer. There are normally atleast three developers in the chain with the last one is the the application developer.

The second Developer in the chain is the specialist API developer with specific detailed knowledge of specific algorithm, he has to provide provide informative Interface for the Application Developer. The API Developer needs to be familiar with the use of Threads. An example of threads can be found in APIs which do storage and retrieval of data from a database aka CRUD operations, with multithreaded database updates, avoiding deadlocks and resource starvation etc. Another example is the progress bar while the application loads in background in a another thread.

The third link is the firmware developer. He is the one who writes drivers, usually in assembly language sub routines, direct calls to hardware. The firmware subroutine(s) is specific to hardware like printers. That is why you have to install drivers everytime you buy a new printer. The firmware provides interface for Operating Systems. The Java JVM is on par device drivers. Good API developers or even Application Developers will interface with and make calls to assembly language subroutines. Not so good ones will interface with Operating System utilities. Given the example Apache Spark, Despite the fact my machine is connected to a router, connected to both Internet and Intranet. Apache Spark could not find the machine IP address. This is because Apache Spark Framework code was trying to locate IP address by using the Operating System utilities. Apache Spark like Hadoop also use SSH for its distributed computing. SSH is another Operating System utility. So in this case the Application is the fourth rung on the ladder. Apache Spark is also coded using a Scala API in Java for this reason the Application Developer is pushed up one rung.

My 68K Assembly Language College Lecturer, who was entering retiring age, remarked once, he doesn't understand why more people don't use Assembly language for building systems. The example he gave was how and one other colleague developed a traffic light system using purely assembly language. He and his colleague were the only rung on the ladder if you don't count the hardware engineer.

Skills like threads and firmware can be taught, moreover have to comply with stated requirements, in the case of firmware developers, the stated requirements are set by hardware engineers. Frameworks like Apache Spark also have preset requirements from predefined mathematical formulas. The Application developer's role is much more complex and more creative. The Application developer must be able to conceptualise the component technology, the product Thread and bind to end user requirements. Application developer has to find his own way and must learn an approach to achieve his goals, otherwise can easily find himself lost and confused, insecure because of what he doesn't know. This is what this website is all about, negotiating the maze that is component technologies.

Two years HND course. The HND Course was focused on practical software development in the real world. I was taught Report Writing skills, Project Planning {Project Management Tool}, four programming languages , Pascal , 68k Assembly , C , C++ and RDBMS & SQL. Essentially I was prepared for and made ready for application development. The course was designed by Mr Dimitri. I realise over the years he passed on knowledge about programming languages which helps me to understand how programming languages work.

The first C assignment programme he gave us was to replicate wc , a unix utility. I did the programme and from then on every time I use an application or framework, my third eye start to see how it was written and I know I can do it. I often trick an API to do what it was not intended for it.

Topics not covered in the HND course were distributed applications and Paralellism. Parallelism is used for processing Big Data in particular in the field of Data Mining, Data Science, machine learning. Although I realised the role I wanted for mysef in information Technology, nevertheless I felt I should fill my firmare knowledge gap. I did this with my first employment opportunity at Smiths aerospace and defence division. I also came across software development best practices being followed which are not normally followed in commercial enviroments.

For example have you read that you need a Ph.D for machine learning applications, I have read this many times. Those who make such claims misundertood Booch when he wrote Industrial Strength Software is beyond the intellect of a developer.

Software Crisis: US Department of Defence

GRADY BOOCH; The United States Department of defence (DOD) is perhaps the largest user of computers in the world. By the mid-1970s, software development for its systems had reached crisis proportions: projects were often late, over budget and they often failed to meet their stated requirements. It was evident that the problems would only worsen as software development costs continued to rise exponentially. To help resolve these problems which were further compounded by the proliferation of hundereds of different languages. The DOD sponsored the development of a single, common high order programming language . The winning design was originally called the Green Language (so called because of its team colour code during the competition), and was renamed ADA.Concurrency was excluded from the original language.

DOD reports highlights two issues leading to failures, one is the plethora of programming languages and the second is failing to meet their stated requirements. This means that developers are consistently being asked to learn new languages which offers no advantage but is actually a disadvantage.

The other problem is failing to meet stated requirements. This identies the failure to communicate or involve the Application developers fully in the requirement capturing so the application developers end up building something which does not meet the stated requirements. Another words the Application developer is programming without intuition. According to The CISQ {Consortium for IT Software Quality } the two problems remain unresolved leading to Poor Quality Software.

Successful Approach to Software Development

Every Product at some level unique. I believe a successful approach to application development is to make sure the Application developers know and understand the use case and carry out the System Analysis before any coding. The appraoch should be as if Programming and System Analysis are inseparable, joined at the hip. The developers together pooling their intellect should break down the system components into smaller components. One doesn't have to use UML, can be anything including scrappy pieces of paper with hand drawn diagram. for example with e-commerce applications one should start by indentifying database tables and screen. The application developers should be part of the System Analysis. I hope to one day work in an a project with this approach.

With this approach Application developers will be motivated and it will be like letting out fired up horses out of the starting gate.

Andrew Ng {deeplearning.ai} who specialises in Machine Learning and wants to democratise this field of Computer Science also recognised that trying to learn programming language and the concepts at the same time is not a successful approach. A better approach is to use a high level language like Octave or MATLAB to understand the concepts. Incidently MATLAB the MAT in MATLAB stands for Matrix ( rectangular array of numbers).

How to be a Trillionair

The worst approach is to start with coding or even without a specification in the case of migration. This coding first approach is surprisingly common and favoured by Coders, programmers, outsourcing companies offering to do migration however avoided by professional application Developers. Hence CISQ: [Consortium for IT Software Quality ] Poor quality software cost an estimated $2.84 trillion in the US alone in 2018

Apache Spark have provided the programming Interface in four languages, Java , Scala, Python and R. It is an approach to make it easier for programmers to learn Apache Spark APIs. However without the prequisites knowledge of AI and Machine learning concepts It is a lazy , inexprienced and failed approach to introducing a framework. Any programmer can copy and past some code and run it. But that doesn't mean the programmer has understood the code or the concepts therefore can go on to develop further applications using the framework. The only diffrerence between the application developer and API developer should be that the API developer has taken their time in developing a robust API. Both must have equal understanding of the APIs.

Complex Distributed Application Paradigms Made Simple

This Level O diagram showing the struts of Internet architecure was first published in 1999 by WROX Press. As an Application developer transitioning from Two tier to three Tier. It was critical I identify the relevant components as the first step. That job was made super easy for me with this diagram. Once I had identified the components then I can look for an Application Generator which included those components with a qualified eye.

I also realised that nothing was going to change in the world of Online Transaction Processing, automated Form processing applications, e-commerce, while the Internet architecure remained the same. Things may get smaller, faster,with trillions invested in this architecture nothing could change. Unsurprisingly nothing has apart from some superficial. The only change I expected is the quality of Tools {component technology, Application Generators} would improve with time. become Industrial Strength. I could even afford to take a break knowing I am not going to missing out on anything. Is there a an application generator avaliable at https://spring.io/ with history of Spring 1 , Spring 2 , Spring 3 , Spring 4 , Spring 5 .... with J2EE application server tomcat 1, 2, 3, 4, 5, 6,7,8,9,10 .... That is what I mean.

Level 0 The Intuition Level

Paradigm Intuition . Intuition can never be enough and can never be too much.

Web Server

Application Server

Competing Paradigms

Apache web server application server wrox press

Building complex systems ....

....from scratch is a costly endeavour and one that is almost never successful".

In the commercial sector to build Industrial Strength Applications, component technology also known as application generators used. Commercial Tools such as Ingres Openroad, Oracle Forms , Visual Basic are examples of 4GL Application Generators used in Two tier applications. I have used all three. The name Oracle forms is intuitive of its purpose, that these types of Tools are used to automate paper forms another words commerce applications. Add an E to commerce then you need another kind of Application Generators. Booch describes programming using 4GL application generators as tedious . I suppose compare to weather monitoring system they are. Using 4GL application Generators will contribute to a successful project however we must not forget the other issue identified by DOD report meeting the stated requirements. Application Developers are not code generators they need to know and understand the Application requirements in order to meet them. Missing out on any one of the two ingredients, the endeavour will end up costly and unlikely to be successful.

Obstacles to Commercial Frameworks

At the beginning of each employment I was sent on a very expensive one week training courses. In my first job employed as a junior software engineer I was sent on a week course on ADA. In my second Job at bytel in slough UK I was sent on a week's course on OpenRoad then later Oracle Forms. I believe some employment agencies offered free training on these 4GLs and that was enough to be employed as an Application Developer. Until then it appeared the way to access employable skills was to go to these expensive Product Training courses until I found Apache Webserver Apache and Tomcat an application server for application developer. I could literally build a Website hosting Company and E-commerce websites with these Tools, without any investment. The two products did not provide enough examples so I was able to buy a book which showed how to build applications.

This find of Open source Software was very exciting for me. Although I was very impressed with Tomcat it was freeware and couldn't be sure of it having commercial value. Then I got hold of a professional J2EE compliant application development Tool. To my surprise IBM Websphere worked exactly the same way as tomcat. It appeared that both of these application development Tool was built from same specification. So there was commercial value in Apache Tomcat. To check if a freeware tool has value, look for a commercial version of it.

These finds came about when I had five years of purely development experience, so I knew exactly what skill level I need to be at to meet the commercial standard. I am also able to pick up any framework without any trainer or supervision because of my approach of Intuition first.

Access to Free Industrial Strength Software

The best Surprise I had while working in Software was to find a free and open source application server called Tomcat. To make sure my find was Industrial Strength I ran a test against a commercial Microsoft browser. I started up the server then typed the url http://localhost:8080. I couldn't believe it, the two pieces of software were compatible.They shook hands. I then compared it to a commercial servlet engine from a canadian company, I had just trained on. A week trainig course. It even came without an IDE. I also found a free database called mysql which was as good as any commercial database including Oracle. Basically I had all the tools I needed completely free to build Industrial E-commerce Applications. Next step is to hide in a cellar for a month and train on the application server with a book until I reach the programmer's plateau.

Who wants to be Billionaire ?

There is a startup company in Australia called CANVA. Their ethos is to make it as simple as possible to build websites directly by the business owner.

The company is worth 3 Billion Dollars for making the complex into simple. Specifically providing a tool for business owners to build a website in a very simple way. You may be inclined to think that this may make web developers redundant. As the company name CANVA suggests, their focus is soley on the HTML CANVAS TAG and for that a Web Server is adequate.

The application developer's activity is focused with the HTML FORM TAG and its complexity. The HTML FORM TAG is used for E-Commerce Applications and for that an application server is required.



canvas form

E-COMMERCE Application Rationale

The first step for me was to gather any new concepts I did not know about from two tier 4th Generation Language {GL} application Generators . Booch refers to this phase as ZEN and the Art of Motocycle Design. After five years of Software Development I was confident I knew what to look for in a framework more than any one around me. Then I would find a free and open source tools which met my requirements to train myself because I was also confident I could do a better job than be trained by a step by step product trainer. A product Trainer's goals are different from mine.