### Founder BackButton Movement 14/11/2019 - document.write(new Date().toLocaleDateString());

#### Mission: Master Industrial Strength in Large or Complex Systems

Grady Booch {IBM}: Industrial Strength Software is beyond the intellect of a developer
This is a quote from my HND Computing course recommended text book, Object Orientated Programing Using CPP.

There are two distinct points in this sentence. first point is that there are two types of software Industrial Strength Software and software which is not industrial strength.

Then there is the suggestion Industrial Strength software is beyond the intellect of a developer . But it is the developer who is going to be building the software product, so how is the developer going to do it ? There is a very complex philosophy embedded here.

The book was teaching Object Orientated concepts using C++. C++ is sometimes referred to as C with classes. The intended audience of the book is the application developer and the use of component technology. Industrial Strength is only beyond the intellect if the developer tries this effort on his own. Does it mean Developers should work in teams of two or more ? No it doesn't mean that either. It means there are a chain of developers with each one acting as links. So when developers work as a chain then Industrial Strength Software is within the intellect of the application developer. There are normally atleast three developers in the chain with the last one is the the application developer.

The second Developer in the chain is the specialist API developer with specific detailed knowledge of specific algorithm, he has to provide provide informative Interface for the Application Developer. The API Developer needs to be familiar with the use of Threads. An example of threads can be found in APIs which do storage and retrieval of data from a database aka CRUD operations, with multithreaded database updates, avoiding deadlocks and resource starvation etc. Another example is the progress bar while the application loads in background in a another thread.

The third link is the firmware developer. He is the one who writes drivers, usually in assembly language sub routines, direct calls to hardware. The firmware subroutine(s) is specific to hardware like printers. That is why you have to install drivers everytime you buy a new printer. The firmware provides interface for Operating Systems. The Java JVM is on par device drivers. Good API developers or even Application Developers will interface with and make calls to assembly language subroutines. Not so good ones will interface with Operating System utilities. Given the example Apache Spark, Despite the fact my machine is connected to a router, connected to both Internet and Intranet. Apache Spark could not find the machine IP address. This is because Apache Spark Framework code was trying to locate IP address by using the Operating System utilities. Apache Spark like Hadoop also use SSH for its distributed computing. SSH is another Operating System utility. So in this case the Application is the fourth rung on the ladder. Apache Spark is also coded using a Scala API in Java for this reason the Application Developer is pushed up one rung.

My 68K Assembly Language College Lecturer, who was entering retiring age, remarked once, he doesn't understand why more people don't use Assembly language for building systems. The example he gave was how and one other colleague developed a traffic light system using purely assembly language. He and his colleague were the only rung on the ladder if you don't count the hardware engineer.

Skills like threads and firmware can be taught, moreover have to comply with stated requirements, in the case of firmware developers, the stated requirements are set by hardware engineers. Frameworks like Apache Spark also have preset requirements from predefined mathematical formulas. The Application developer's role is much more complex and more creative. The Application developer must be able to conceptualise the component technology, the product Thread and bind to end user requirements. Application developer has to find his own way and must learn an approach to achieve his goals, otherwise can easily find himself lost and confused, insecure because of what he doesn't know. This is what this website is all about, negotiating the maze that is component technologies.

Two years HND course. The HND Course was focused on practical software development in the real world. I was taught Report Writing skills, Project Planning {Project Management Tool}, four programming languages , Pascal , 68k Assembly , C , C++ and RDBMS & SQL. Essentially I was prepared for and made ready for application development. The course was designed by Mr Dimitri. I realise over the years he passed on knowledge about programming languages which helps me to understand how programming languages work.

The first C assignment programme he gave us was to replicate wc , a unix utility. I did the programme and from then on every time I use an application or framework, my third eye start to see how it was written and I know I can do it. I often trick an API to do what it was not intended for it.

Topics not covered in the HND course were distributed applications and Paralellism. Parallelism is used for processing Big Data in particular in the field of Data Mining, Data Science, machine learning. Although I realised the role I wanted for mysef in information Technology, nevertheless I felt I should fill my firmare knowledge gap. I did this with my first employment opportunity at Smiths aerospace and defence division. I also came across software development best practices being followed which are not normally followed in commercial enviroments.

For example have you read that you need a Ph.D for machine learning applications, I have read this many times. Those who make such claims misundertood Booch when he wrote Industrial Strength Software is beyond the intellect of a developer.

### Software Crisis: US Department of Defence

GRADY BOOCH; The United States Department of defence (DOD) is perhaps the largest user of computers in the world. By the mid-1970s, software development for its systems had reached crisis proportions: projects were often late, over budget and they often failed to meet their stated requirements. It was evident that the problems would only worsen as software development costs continued to rise exponentially. To help resolve these problems which were further compounded by the proliferation of hundereds of different languages. The DOD sponsored the development of a single, common high order programming language . The winning design was originally called the Green Language (so called because of its team colour code during the competition), and was renamed ADA.Concurrency was excluded from the original language.

DOD reports highlights two issues leading to failures, one is the plethora of programming languages and the second is failing to meet their stated requirements. This means that developers are consistently being asked to learn new languages which offers no advantage but is actually a disadvantage.

The other problem is failing to meet stated requirements. This identies the failure to communicate or involve the Application developers fully in the requirement capturing so the application developers end up building something which does not meet the stated requirements. Another words the Application developer is programming without intuition. According to The CISQ {Consortium for IT Software Quality } the two problems remain unresolved leading to Poor Quality Software.

### Successful Approach to Software Development

Every Product at some level unique. I believe a successful approach to application development is to make sure the Application developers know and understand the use case and carry out the System Analysis before any coding. The appraoch should be as if Programming and System Analysis are inseparable, joined at the hip. The developers together pooling their intellect should break down the system components into smaller components. One doesn't have to use UML, can be anything including scrappy pieces of paper with hand drawn diagram. for example with e-commerce applications one should start by indentifying database tables and screen. The application developers should be part of the System Analysis. I hope to one day work in an a project with this approach.

With this approach Application developers will be motivated and it will be like letting out fired up horses out of the starting gate.

Andrew Ng {deeplearning.ai} who specialises in Machine Learning and wants to democratise this field of Computer Science also recognised that trying to learn programming language and the concepts at the same time is not a successful approach. A better approach is to use a high level language like Octave or MATLAB to understand the concepts. Incidently MATLAB the MAT in MATLAB stands for Matrix ( rectangular array of numbers).

### How to be a Trillionair

The worst approach is to start with coding or even without a specification in the case of migration. This coding first approach is surprisingly common and favoured by Coders, programmers, outsourcing companies offering to do migration however avoided by professional application Developers. Hence CISQ: [Consortium for IT Software Quality ] Poor quality software cost an estimated \$2.84 trillion in the US alone in 2018

Apache Spark have provided the programming Interface in four languages, Java , Scala, Python and R. It is an approach to make it easier for programmers to learn Apache Spark APIs. However without the prequisites knowledge of AI and Machine learning concepts It is a lazy , inexprienced and failed approach to introducing a framework. Any programmer can copy and past some code and run it. But that doesn't mean the programmer has understood the code or the concepts therefore can go on to develop further applications using the framework. The only diffrerence between the application developer and API developer should be that the API developer has taken their time in developing a robust API. Both must have equal understanding of the APIs.

### Complex Distributed Application Paradigms Made Simple

This Level O diagram showing the struts of Internet architecure was first published in 1999 by WROX Press. As an Application developer transitioning from Two tier to three Tier. It was critical I identify the relevant components as the first step. That job was made super easy for me with this diagram. Once I had identified the components then I can look for an Application Generator which included those components with a qualified eye.

I also realised that nothing was going to change in the world of Online Transaction Processing, automated Form processing applications, e-commerce, while the Internet architecure remained the same. Things may get smaller, faster,with trillions invested in this architecture nothing could change. Unsurprisingly nothing has apart from some superficial. The only change I expected is the quality of Tools {component technology, Application Generators} would improve with time. become Industrial Strength. I could even afford to take a break knowing I am not going to missing out on anything. Is there a an application generator avaliable at https://spring.io/ with history of Spring 1 , Spring 2 , Spring 3 , Spring 4 , Spring 5 .... with J2EE application server tomcat 1, 2, 3, 4, 5, 6,7,8,9,10 .... That is what I mean.

### Level 0 The Intuition Level

Paradigm Intuition . Intuition can never be enough and can never be too much.

### Building complex systems ....

....from scratch is a costly endeavour and one that is almost never successful".

In the commercial sector to build Industrial Strength Applications, component technology also known as application generators used. Commercial Tools such as Ingres Openroad, Oracle Forms , Visual Basic are examples of 4GL Application Generators used in Two tier applications. I have used all three. The name Oracle forms is intuitive of its purpose, that these types of Tools are used to automate paper forms another words commerce applications. Add an E to commerce then you need another kind of Application Generators. Booch describes programming using 4GL application generators as tedious . I suppose compare to weather monitoring system they are. Using 4GL application Generators will contribute to a successful project however we must not forget the other issue identified by DOD report meeting the stated requirements. Application Developers are not code generators they need to know and understand the Application requirements in order to meet them. Missing out on any one of the two ingredients, the endeavour will end up costly and unlikely to be successful.

### Obstacles to Commercial Frameworks

At the beginning of each employment I was sent on a very expensive one week training courses. In my first job employed as a junior software engineer I was sent on a week course on ADA. In my second Job at bytel in slough UK I was sent on a week's course on OpenRoad then later Oracle Forms. I believe some employment agencies offered free training on these 4GLs and that was enough to be employed as an Application Developer. Until then it appeared the way to access employable skills was to go to these expensive Product Training courses until I found Apache Webserver Apache and Tomcat an application server for application developer. I could literally build a Website hosting Company and E-commerce websites with these Tools, without any investment. The two products did not provide enough examples so I was able to buy a book which showed how to build applications.

This find of Open source Software was very exciting for me. Although I was very impressed with Tomcat it was freeware and couldn't be sure of it having commercial value. Then I got hold of a professional J2EE compliant application development Tool. To my surprise IBM Websphere worked exactly the same way as tomcat. It appeared that both of these application development Tool was built from same specification. So there was commercial value in Apache Tomcat. To check if a freeware tool has value, look for a commercial version of it.

These finds came about when I had five years of purely development experience, so I knew exactly what skill level I need to be at to meet the commercial standard. I am also able to pick up any framework without any trainer or supervision because of my approach of Intuition first.

### Access to Free Industrial Strength Software

The best Surprise I had while working in Software was to find a free and open source application server called Tomcat. To make sure my find was Industrial Strength I ran a test against a commercial Microsoft browser. I started up the server then typed the url http://localhost:8080. I couldn't believe it, the two pieces of software were compatible.They shook hands. I then compared it to a commercial servlet engine from a canadian company, I had just trained on. A week trainig course. It even came without an IDE. I also found a free database called mysql which was as good as any commercial database including Oracle. Basically I had all the tools I needed completely free to build Industrial E-commerce Applications. Next step is to hide in a cellar for a month and train on the application server with a book until I reach the programmer's plateau.

### Who wants to be Billionaire ?

There is a startup company in Australia called CANVA. Their ethos is to make it as simple as possible to build websites directly by the business owner.

The company is worth 3 Billion Dollars for making the complex into simple. Specifically providing a tool for business owners to build a website in a very simple way. You may be inclined to think that this may make web developers redundant. As the company name CANVA suggests, their focus is soley on the HTML CANVAS TAG and for that a Web Server is adequate.

The application developer's activity is focused with the HTML FORM TAG and its complexity. The HTML FORM TAG is used for E-Commerce Applications and for that an application server is required.

### E-COMMERCE Application Rationale

The first step for me was to gather any new concepts I did not know about from two tier 4th Generation Language {GL} application Generators . Booch refers to this phase as ZEN and the Art of Motocycle Design. After five years of Software Development I was confident I knew what to look for in a framework more than any one around me. Then I would find a free and open source tools which met my requirements to train myself because I was also confident I could do a better job than be trained by a step by step product trainer. A product Trainer's goals are different from mine.

• Backbutton {Form Duplication submission; Idempotency }
• Security
• Direct URL Access
• DB Connection Pooling
• SQL Flexibility
• Internationlisation
• session management in stateless environment
• Push capability in a pull model architecure
• Stable feature Rich Informative IDE
• Debugging Capabilities
• Disk Space Full during CRUD operation -> System Crash
• A Framework should show good programming techniques
• Error Handler

### Error Handler

These are examples of Error Handling mechanisms. The framework's Error Handler's intended audience is the Application Developer.
Although this message may not mean anything to an end user It is enough informatiion for the Application developer to fix without needing to contact support. The Application Error Handling mechanism provided by a framework for the Application Developer. The end user can fix the issue without needing any support.
These frameworks are examples of Industrial Strength Software, worth investing a developer's time and effort in training.

### Application Error Handler

#### Poor Quality Framework Error Handler

Now that I have shown you two Industrial Strength error handlers here is a comparison to a really pathetic, disasterous Framework error handler which is NOT Industrial Strength.
I came across this example of Poor quality software from Apache Spark.
The following code is the simplest code you can run on Apache Spark.
scala> spark.range(4).reduce(_+_)

I ran this code in Apache Spark version 3.1.0.
It create column with the column name of id and populates it with sequential values from 0-3. Then sums all the elements

Instead of returning the the answer of 6 the apache spark returned this error.

:24: error: overloaded method value reduce with alternatives:
(func: org.apache.spark.api.java.function.ReduceFunction[java.lang.Long])java.lang.Long
(func: (java.lang.Long, java.lang.Long) => java.lang.Long)java.lang.Long
cannot be applied to ((java.lang.Long, java.lang.Long) => scala.Long)
spark.range(4).reduce(_+_)
^

I logged the error on the Apache Spark user mailing list and I got no response.
So then I decide to conduct an investigation and do some tests. I separated the two Spark functions, first ran the range function using a variable. The console automatically displayed the result.

scala> val myrange = spark.range(4)
myrange: org.apache.spark.sql.Dataset[Long ] = [id: bigint ]

This showed the anomoly, I have highlighted in red.
I then ran the collect () function against the dataset.
scala> spark.range(4).collect()
res20: Array[Long] = Array(0, 1, 2, 3)

The collect() function is normally used to display data in the dataset. I noticed the collect() also transformed the data type of the dataset to an array of elements type Long. So as a hunch I inserted the collect() between spark's range() function and reduce() function. This had the effect of piping the output of one function and redirects to another function. Programmatic version of unix or linux pipe | utility. That solved the error and confirmed there is a BUG.

scala> spark.range(4).collect.reduce(_+_)
res0: Long = 6

The dataset API is literally Apache Spark's core API, Spark's foundation API.
scala> myrange.show
+---+
| id|
+---+
| 0|
| 1|
| 2|
| 3|
+---+

Once I emailed back the result of my experimentation and proved the BUG, only then did Spark developer responded.
Can you see any mention of bigint in the error message ?
This framework error handler is the most dumbest and inaccurate error handler I have ever seen.
It goes to show that the product support developers did not respond because they do not understand their own error messages more over refuse to acknowledge it was a BUG.
I have a deep understanding of programming so I was able to track and trace the problem using my collection of tips and tricks. Do you have the same deep understanding and will you be able to find the BUG ?
the same code does actually work on earlier versions of Apache Spark.

### Booch: Prototype!, Prototype! ,Prototype!

So now you have found the application Generator with all the concepts. let's say you only have access to its interface but not the implementation details. The components can react unexpectedly to your environment. Each vendor has invested huge sums of money in their frameworks. Occasionally Tool Vendors may download Open Source and make it Industrial Strength. There are two ways to choose a framework, one is to read the reviews and the other is Booch's way. Booch says this not once, not twice but three times, PROTOYPE , PROTOTYPE, PROTOTYPE.

Prototyping will also give you much needed Hands-on experience and practice. If during the Protoyping phase, especially with Open Source Tools, you find some issues with the framework i.e. bugs, Most open source software have mailig lists. You can log any anomolies you find on StackOverflow or as JIRAs. The commercial vendor(s) monitors these and fix the bug in their commercial product.

### OLTP Connection Pooling , Load Balancing

The Fat client Users are trained users. Trained to use the screens, trained so they know what are the required values for the input fields. Never the less you still provide drop down lists when ever you can, to ensure data stored in the database is clean. The fat client user is likely talking to the customer at the same time while operating the software, so for ease of use the fields are assigned tab sequences. Tab Sequence is a feature of 4GL Application Generators which allows the user to go next field using the TAB keyboard key, so the operator doesn't have to reach the mouse.

The e-commerce user on the other hand is an untrained user. When the user lands on the web page, You need to need to make sure the website is easy to understand and simple for the user to buy the product or service. The very first consideration for your untrained user is simplicity of web page. Avoid using text. The good news of course is that as a developer you are only using the basic features and you only need to be familiar with the basics.

Another consideration for an e-commerce website is response time. In the case of fat client the end user will tolerate it, but if same happens with e-commerce site, the consumer may think the website is broken. So you may lose business.

There will be a peak period. This is likely to be in the evening between 6pm -10pm, when the consumer is looking to do his internet shopping. During this peak period demand find a solution so that performance is not effected. The solution I came up with Java's Remote Method (RMI) Invocation technology integrated with the application server. So that the front application is not burdened with all the processing

The backbutton problem where the user can on an e-commerce website make duplicate orders or make duplicate payments. This is not an issue with 4GL application Generators.

These problems need to be solved by the framework.

### Database Connectivity

Theses two examples of Component Technology code are provided by Tool Vendors and cannot be learned from learning a programming language. The decisions as to which component technology should be used should to be made by System Architect(s) when protoyping applications then the strategy delegated to application developers to follow and not left to individual application developer to come up with their own strategies.

### Idempotency

Safeguards from duplicate Transactions

### The Concept, the Archiecture of the DAPI

The clutter of stub and Skeleton was later removed leaving behind, the pure Objected Orientated Concept of The Interface. I think this is the purpose of the Interface, It was meant to be used for the [DAPI] Distributed API just as the purpose of the class is Encapsulation.

This is also an excellent example of Component technology improving with iterative development, where the duplicate code of stub and skeleton was removed.

You will not need Operating System utilities like SSH installed as a prerequisite nor etc/hostnames,etc/host text file to be populated with machine IP address to use RMI for distributed computing, that's because RMI is Industrial Strength Software.
To use the RMI's remote feature one simply applies couple of lines of code in the client like ;
SomeInterface si = (SomeInterface) Naming.lookup ("//SomeHost[:1024-65535]/MyServer");
String retVal = si.SomeMethod("coco");

### The Protoype using Java RMI API

The messages "coco [0..4] is trying to contact!" is a working demonstration of the compute engine or server side processing, The work is off loaded from the application server. That is something you can't do with 4GL Application Generators like OpenRoad, Oracle Forms. When I realised this, it was time to abandon that ship and shed those outdated skills.

retVal =

The Best Practice

The code.
The RemoteException is a Java checked Exception so the omission of this will give a compile time error. It is a Best Practice for socket programming. The history of this goes back to IBM and MandelBrot, the creator of MandelBrot Factorial Pattern. If like Apache Spark you use Operating System utility in the form of C DLLs for framework development then you may not have access to this or may not be aware of it.

### Variable Scope

It is my view that one needs a mechanism where data which can be passed around by forms {JSPs}. I am not referring to the URL parameters which can pass data from form to form. To perform this task a mechanism called jsp:useBean can be used. Another word for implementation and tracking of variable Scope use JSP Bean TAG. This application server mechanism allows data to be passed around at four levels, the page , request , session and application. That is to say the objects are persistent at four levels and any remaining data objects are destroyed when application is shutdown. The Enterprise Java Bean EJB appears to performs the same function but with Servlets . So if you make your priority understanding the concepts before hand then you are in a position as Application on which component to focus on and find, otherwise there is a danger of the headless chicken syndrome. I feel taking this route of concepts first is where you can be secure in what you know and what you don't know you wisely avoided knowing.

### Level 0 - OLAP CUBE {Business Intelligence}

The OLAP CUBE component technology can also be categorised as Data Science. OLAP came before data science. It was known as just OLAP CUBES or data warehousing. I was working as a developer under a product manager. He asked me to look in this with a view to apply it to the TELCO database. The concepts are simple enough. Mostly a select statement with aggregation of data using a group by and a where clause. Once I figured out the concepts, I then tasked the Product Manager to get hold of a Tool which will allow me to build some cubes which will give intuition from the 250 Table Database. The business Managers can use the Cubes (drilling from graphical cubes) hep them identify customer trends therefore make better business decision. The product Manager didn't know any concepts so he wasn't able to provide me with a Tool.

### Level 0 - Machine Learning Algorithm Intuition Diagram

To see the intuition of Data Science you just need to put together these two Level 0 diagrams. Machine Learning technology is truly versatile, innovative and far more exciting than the tedious formentioned Form frameworks. There are claims that one needs a ph.D to be able to use Artificial Artificial or Machine Learning. Component technology must be unfamiliar territory for those who hold the view one needs a ph.D. Frankly it is just another component technology with daily new supply of APIs. To develop the actual API one may need ph.D knowledge. The API developer needs to provide an interface which meets the expectations of the Application Developer t meet his use case requirements. So I would say the ph.D API developer needs the right software skills. Obviously as an Application developer, one needs to start with an understanding of the concepts, use cases, in order to utilise the Machine Learning APIs.

### Level 0 - Multi-purpose Application Generator using Artificial Intelligence

This is one of my favourite component technology. It is the work of a single developer over a period of ten years. It is very well documented with use cases. All ready for a developer to come along and apply it. For example a single developer can build an Industrial Strength last mile product leading to huge cost saving for any company not forgetting cost saving in software development .

### Essence of Machine Learning

#### The Three Rs

When you choose to train from the best you will be the best from Day one.
Needing years of commercial experience to be good at using technology is a myth cultivated by those who have never known the pleasure of the intuition. There is a minefield of trainers, training companies, and not forgetting copy and paste books.

Alot of these training companies are following the step by step approaches in a parrot fashion leaving out pieces,failing to explain those pieces they do not understand so the student doesn't understand.

With this approach I will know the concepts going into an Industrial Strength machine Learning algorithms of the APIs. This approach will take a lot longer than most other short cuts. This is my preferred approach. I want the complete understanding. The more difficult the stronger feeling of achievement.

The mathematical formula which has taken years if not centuries to develop. I first came across this type of maths when I was 14 years old. Unfortunately I have lost my maths jotters. I believe it was referred to as simultaneous equations.

#### The Code

The mathematical formula translated into code,
Not the complete code.

#### The Algorithm Goal

This is what the Gradient Curve should look like , heading towards the global minimum when it is trying to reduce the Loss Error, therefore this is an Industrial Strength Gradient Descent. When the curve flattens out this is when the unknown values of the Coefficient parameters have been found.

#### Map of mathematics

This diagram can help to solve the problem of Jack of all trades and Master of none, where you can become the master of the chosen one, without having to explore the irrelevant areas of maths with perhaps a ph.D. Make the best use of Component Technology as Application Developer to manufacture products for the best interest of humanity.

This code is an example implementation from the field of Numerical Analysis and from the first book written about algebra written in Persia (Iran) circa 820 during the golden age of Islam. Please see the diagram above, the Green circle in the purple square and the black dots representing darts for super easy Intuition.

#### Pi Estimation complete Intuition

The theory is that if you put a circle in a square, throw enough darts, compare the number of darts in the circle with number of darts in the square proportions, you can approximate the value of Pi. Numercial Analysis requires powerful computers. Hence this is the front page Apache Spark hands-on example. It is important to understand this method of estimatig Pi is purely an academic mathematical exercise. It is designed to test performance of Super computer processing power as a single super computer where the operating system manages each task across the stack of motherboards, or with paralellism where there is a software solution managing the task across multiple smaller computers connected by Network cable(s) acting as one virtual super computer. In parallelism the data is divided into smaller chunks and the logic is processed simultaneously on the smaller chunks of data. The underlying reason for this exercise is to get the performance measurement which is the time taken to carry out the high amount of computation. It is neither an optimisation nor a convergence problem. An example of optimisation is buying an item that is value for money. Paying the least to get the most. An example of Convergence is a gradient descent mathematical formula in machine learning, where the objective is to find accuracy with computations. In this exercise the idea is to cover the whole area of the circle and the square with randomly generated x,y points/darts marks to calculate π.

concretely: when the board is centered at (0,0), and that r=1 (the radius of the circle) then diameter is 2, therefore side of square is also 2, and area of square = 22 = 4.

Keep in mind that each dart can land in (x∈[−1,1],y∈[−1,1]) and that a dart that lands at (x,y) falls within the circle if

$\sqrt{{x}^{2}+{y}^{2}}<1$

Because each throw falls randomly within the square, the probability of a dart landing within the circle is given by the ratio of the circle’s area to the square’s area:

${P}_{circle}=\frac{Are{a}_{circle}}{Are{a}_{square}}=\frac{\pi {r}^{2}}{\left(2r{\right)}^{2}}$

we can interpret Pcircle as being approximated by the fraction of darts thrown that land in the circle. Thus, we find: $\frac{{N}_{circle}}{{N}_{total}}\approx \frac{\pi {r}^{2}}{\left(2r{\right)}^{2}}=\frac{\pi }{4}$

To solve for π = 4 x Ncircles/Ntotal . This is the math formula implementated in the Apache Spark Python Code. Do keep in mind the answer π is a running a Estimate.
Or as my school teacher used to say change the side change the sign.
What you do to one side you must do to both sides.

With the intuition I think it is quite obvious the Apache Spark Code is NOT a Good demonstrated of Good Programming Techniques by a framework, on the contrary a poor example.

For this formula to have confidence means the two areas need to be densely covered with darts. Otherwise the answer can be easily skewed. To fully cover the two areas with darts one needs to carry out high amount of computation.

If you throw one dart then the answer is (4 x 1/1) 4 or 10 (4 x 100/100) darts and they all land in the circle then the answer is also 4. 4 is close to 3.14 but now you should now understand see arriving at approximate Pi of 4 which appears to be close to 3.14 is actually Pointless. Therefore setting num_samples = 100 is coding without intuition.

What is the randomised probability of approximately 22% darts landing in the square with about 78% darts landing inside the circle after throwing 100 darts ?
The starting number should be NUM_SAMPLES = 108.

### Coding without Intuition

My motivation for providing a detailed intuition for the six lines of code apache Spark code is to show the inenvitability of failure to meet the requirements when codingand or testing without the rationale. Failure to meet the stated requirements is the other DOD finding as to the cause of software crisis. Clearly the problem stil exists. Poor quality software cost an estimated \$2.84 trillion in the US alone in 2018 . I have come across both coding and testing without intuition many times. It is a common misconception especially in component technology, because there is less code a developer will arrive at the intuition from the code. In component technology one can easily get into complex rabbit holes . This example shows how wrong the approach is when coding without intuition.

Jillur Quddus, the author, is described as Technical Lead Architect,polyglot software Engineer and data scientist with over 10 years of hands-on experience in architecting and engineering distributed, scalable , high performance and secure solutions used to combat serious organised crime, cybercrime, and fraud. Jillur has extensive experience in working within central government , intelligence , law enforcement and Banking, and has worked across the world including in Japan , Singapore , Malaysia, Hong Kong and New Zealand. Jillur is both the founder of Keisan , a UK based company specialising in open source distributed technologies and machine learning , and the technical lead architect at Methods, the leading digital transformation partner for the UK public Sector.

Step 1
In step 1 Jillur advocates importing dependencies. He says to setup an Operating System environment variable SPARK_HOME then use findspark.init() to locate Spark installation directory. If one uses this method then one is literally hardcoding to the machine.
I would call this step the initialisation block. This is where the programmer PROGRAMMATICALLY initialise any needed resources. The correct method is to overload the method as findspark.init('path/to/spark'); This way another programmer can use the same code and if the Spark installation path is different then the programmer can see it.

Step 2
As you can see from the above Apache Spark Code there are three tabs showing the code in three languages. Scala, Java and Python. The code shown in the book can be run as Java or scala but cannot be run as Python because the forward slashes are missing. Jillur has copied and pasted the wrong code. It is not tried or tested code by a hands-on Programmer. Jillur describes himself as a hands on programmer.

Step 3
Step 3 failure is also identified in the DOD report. It is a failure to meet the stated requirements. I have explained the stated requirements fully in the intuition section above. Jillur does not have the intuition of the programme so he has set the num_samples = 10. Setting the num_samples=10 doesn't test for the compute intensity performance, which is the requirement. The failure to provide full intuition of the code is as much a failure of Apache Spark . This failure to detail intuition shows that even 6 lines of code and its functionality can be misunderstood hence the failure to meet the stated requirement.

Step 4
The result print(pi). It is diffcult to get this wrong , but still manage to show poor quality by missing the original description and formatting ."Pi is roughly %f"

Step 5
Step 5 is the cleanup block of code.
Jillur has copied the sc.stop to clean up the started server. Had he programmatically setup the environment he could have cleaned up the environment. As things are the environment will be left with a residue, causing future bugs for not only this machine but for others who ran this code on other machines.

On page 62 Jillur described this exercise as integrating Jupyter notebook with Pyspark. Actually that is a completely different setup which does not require steps 1 and 2.

### Career Path

#### Junior Software @ Engineer Smiths Aerospace and Defence Division

Rather than accepting a safe career job as product support, I began my career working as a junior software Engineer in a safety critical bespoke hardware and software development environment. The programming language used on this project was ADA.

The development staff comprised of Tech Lead, Hardware Engineer, Assembley language programmer , mathematician. I joined with two others at the testing stage.

##### Product Intuition
This was the final phase of testing, unit testing, I think it was the final client requirement that needed to be met. So The tech Lead had nothing to do but wait for the results of unit tests and the project to be formally declared finished.

I approached the tech lead and asked if he had a minute ?. He replied yes sure. I know I am supposed to do unit testing, I look around there is so much going on, the big Rig test room and all these documents, Do you have a top level document how the system works. He wrapped his sandwitch up and put it away neatly in his lunch box. Pulled an A4 sheet from one his document bundles. Flipped the page over to the blank side. Picked up the bic pen and began to draw. After he had drawn a number of components he then began to explain each of the components. He explained it so clearly that I was asking questions as he was explaining and he was giving answers. He shocked me a bit with one detail. When he was finished I asked him that I heard that aeroplane software has black box and the software is supposed to be fail safe. He pulled his chair back and said here is the black box. He asked him why do you have the black box under your desk. He said he signed off for it and if he loses it he will in trouble because it is worth more than his house.

I then asked if there were any projects in c because I am good at c, I like c. He replied don't let anybody hear you say that word again, then started to make spitting gestures. He went on to say look what you made me do you made me say that word which has left a bad taste in my mouth. I asked what word ? he replied C and started to make spittig gestures again. I apologised :) I knew he was joking with me but at the same time making a serious point. He also explained how the bespoke software was fail safe proofed. It took all of 20 minutes I had the intuition. The System was demistified.

After that day my best time of day was as I came in to work. The moments when I walked past the Iranian ph.D student who was doing his thesis on the project. He was going to spend a year writing his thesis from the specification, test documents etc. Searching through Bundles and Bundles of documents. He was always holding his head and always looked stressed trying to figure out the system so he can document it. I on the other hand accomplished the same in 20 minutes with a Intuition diagram from the project Technical Lead who developed the system over the past 4 years. The diagram labelled "competing paradigms", means so much to me because it reminds me of the diagram the Technical Lead drew for me.

I was then working under a contracor who told me how to do unit tests. He told me to create stubs and pass arguments to each function. That did not make sense to me as testing anything worth testing. I clarified this with the tech lead on the contractor's day off. The tech Lead said you are not supposed to be doing tests like that , you are supposed to test from the specification.

I think that was the first person I got fired. I got all but one of the Indian outsourcing team sent packing to India as well @ bytel , after I told the manager about the OMG assessment by the senior contractor when coding without intuition, and a whole lot of students hired to work on individual modules, that's another story.

I was moved off the testing and was sent to fix a bug on an aircraft monitoring system. The system had gone live and the senior Tech Lead on that project left as the project was signed off and complete. I did fix the bug. Moving from unit testing to maintenace is a promotion and a recognistion I had a deep understanding of code. I gained the deep understanding after I did an accountancy project in C during my HND course. I was able to do that because I had accountancy intuition and by this I gained programming conceptualisation.

Finding the Bug required a code walk through. It is important to remember that it was possible with this project because third party APIs are NOT PERMITTED. So no hidden concepts in API component technology. It is raw pascal and emdedded 65k Assembly code. I had learned Pascal during my HND course. It was the first programming language I learned. I just didn't get programming. At this point however I had learned five languages so I had no problem with tracing. The Pi estimation example comparatively is API component technology and hides many concepts, there I have demonstrated the results of code without intuition.

reason for leaving: A new contract by Boeing to build a 500-600 seater aircraft was cancelled, The Senior Boeing Engineer revised the cost, it was judged too expensive without customers like British Airways making advance payments to off set initial Research and development cost. Airbus did later build that aircraft.
Duration 1 years.

#### Application Developer @ bytel , Slough UK

I then moved single product developed using a 4GL Ingress OpenRoad. The product was developed by following every best practices leading to its success Generating £18 GBP Million Pounds Revenue per annum from the work of 10 to 11 4GL developers, 2 testers, 3 Data Analyst Headed by a world authority on Databases. The product had 250 tables on Ingres database. Interestingly all developers were contractors with each one specialising in a module of the Telco product.
The first week I was trained on the product as user along with customer users , and bytel support. The product also had user manual never the less a trainer trained every one explaining the user manual. This is a best practice, because with this approach the developer is given a intuition.

Then I was sent on a week of 4GL OpenRoad Training course. Later when a decision was made to migrate from Open Road to Oracle Forms I was again sent on a week's course. Booch refers to these frameworks as tedious application generators because they are limited in what they can do and are both competing for space as two tier architecures.

The human resource manager arranged two meetings for further skill development. The first one was with the project manager. He explained what an application thread is and that if the three of us focused on just programming we would plateau out in our skills and we would remain as juniors. He said we should try to develop into system analysis. One developer objected to being referred to as junior.

The next meeting was with head of Analysis team. He was known as a world's authority on databases. I think it was easy to be regarded as an expert in such things those days because the ignorance and mystery the technology held for others. He pointed out the benefits of databases. I came out of the meeting with the impression get the database to do everything you can for you.

The company was taken over by American Investors. They decided to migrate from Ingres OpenRoad to Oracle Database and Oracle Forms. Both frameworks are 4GL Application Generators. 18 months later I came across J2EE technology server side architecure. The next migration should have been to use J2EE technology. I knew the product architecure quite well and server side architecure, J2EE is well suited for the Telco OLTP product, by this time I was working at Logsys Solutions, so no need to contact bytel and inform them the outsourcing to India was a bigger disaster then they could imagine.

The migration was outsourced to India and was being carried without intuition of the product certainly without framework technology. Even the world's authority on databases was no help with new technology innovations and architecures

##### OMG! Bug
I was given a bug to fix causing incorrect billing. The system was called SMS customer and Billing System. The issue was in the core module and the most complex module. The manager who gave me the bug also gave me the home number of the senior contractor. He was considered the expert in the system and too important to let go. Each contractor took care of one module, module maintenance and customer requested billable enhancements. So I called the senior contractor for advice. He asked me if I had worked on the billing module before ? I said no. He replied Oh My God they gave you that Module to work on. I understood what he was saying. He knew working with code without Intuition was ridiculous.

All the other contractors had been let go in favour of outsourcing. The contractors were given a week's notice and told to document their modules. I noticed that one developer wrote half a page and left it on his desk on the last friday afternoon before leaving. The line manager who was a hiring manager, knew nothing of technology, picked it pick on the following monday.

What was even more ridiculous is expecting an Indian outsourcing company to work on whole system with thousands of lines of code without intuition. A disasterous approach to software development.

With this attitude of outsourcing to India the company did not want UK trained, education developers but preferred outsourcng to India and developers working without intiuation as code generator so I felt it was better for me to leave. When the project manager referred to programmers who worked without product Intuition as Junior developers he was referring to these outsourced developers.

There was a department for each function i.e. windows support, First line of product support, a database department, Data Analysts department, application developer, testing department. I wanted access to all the technologies and functions of a software development product. That wasn't going to happen at bytel.
Duration 2 years.

#### Software Engineer @ Logsys Solutions

I needed Software Development experience where projects would be at different stages. I found the exposure I wanted at this software house. After only working there for four weeks all the sales department was closed. So the writing was on the wall. I had an objective which was to exposure to all commercial products especially the Oracle Database and that is what I focused on.

Then one day I was asked to go on web development product course. Two of us went while one who was a Visual Basic developer refused to go. He said he had been working on visual basic for four years and he wasn't prepared to transition.

Then a new project came in to the company. The visual Basic Programmer was chosen as the Tech Lead. There were three of us working on the project. I then started programming in Visual Basic without any training. Just learning and developing as I went along. That didn't bother me because Visual Basic a 4Gl.

He told us to do screens using Visual Basic drag and drop, based on his understanding of the specification. I soon realise he was following a screen process flow from a workflow product which allowed one to do rapid application design.

I expected for him to have meetings to discuss the specification and the approach we should take. He did have meetings and all he did there was praise me and how well I was doing with Visual Basic.

Then one day I decided to tell him the bad news. I told him you have chosen Visual Basic to do this project but everything is going server side. He did the smart thing and found another visual basic software maintenance job. Leaving behind an unfinished project he was supposed to be leading. His approach was coding first, a disasterous approach, he can be described as junior developer as defined by the project manager at bytel.

Reason for Leaving: I do not wish to be like the Unit Testing contractor, or Indian outsourced developers or the Visual Basic Tech Lead with coding first approach. my approach is intuition first, this approach was further confirmed as the right approach by OMG response. with this approach I knew all the concepts of Server Side Application Programming. Then I had to get hands-on experience which I would do with Booch's way of protoyping , prototyping , prototyping.
Duration 2 years.

## Miscellaneous Experiences and Findings

#### Original Algorithm of a cash Register; Code Walkthrough

I sneaked the trim() function in this code. It was an inspiration I took from Apache Spark. The Apache Spark framework developer after reading a line of text into a String as the the IP address from the text file etc/hostname then used the trim() function to remove the spaces. It had the effect of coverting the line 127.0.0.1 localhost into 127.0.0.1localhost. Then the Apache Spark generated an error message that the IP address could not be found :) . The framework developer must have copied and pasted the code from the internet and without understanding the code left the trim() function unaware of its consequences. I put the trim() function in my code knowing there are no consequences.
Incidently when you start spark-shell if you see this message WARN Utils: Your hostname, < hostname > resolves to a loopback address: 127.0.1.1; Then Apache Spark is using Operating System network DLLs after looking into /etc/hosts (root user) or etc/hostnames (non-root user) file for IP address.

This is the the way code is written in Safety Critical software where third Party APIs/libraries are not permitted. Ideal code Walkthrough sample because it should be easy to read and follow. If you think I am only using basics therefore not good at programming then you have a knowledge gap in Software development best practices.

Artificial Intelligence is the next generation of architecures for building applications. Similar to three tier Web technology was the technology replacing 2 tier architecures. Although I do not think Artificial Intelligence technologies is the next generation direct replacement for OLTP web applications technologies as J2EE was to Oracle Forms, Visual Basic , Ingres OpenRoad. I think that the misunderstanding or lack of intuition in these technologies will lead to and is leading to misfittings of technologies. OLTP developers need to aquire completely different mindset in order to be able to apply Artifical Intelligence technologies. Application developers need to know when and when to use these technologies. In a nutshell Artificial Intelligence technologies are not Form ware. To get a sure fire feel for this new technology one has to prototype protoype , protoype.

### FAG-A-TUNA

I was sitting outside my office building during my lunch break having a FAG. I used to smoke then. A guy came outside with a cigarette in his hand. I quickly offered him a light before he could take out his lighter. It is a good way to start a conversation. I have not seen him before so I asked If he just started. He said he was here only for two weeks and he was to go to customer sites to TUNE the live running databases. I was taken a back by this because we had three DBAs and they had a department of their own. I didn't know there was even such a thing. I asked him what did TUNING a database involved. He explained to me what he did and they all led improving application speed and performance.

### Coding Standards

This is the NASA's Rule of 10 Coding standards. One rule is to turn on compiler warnings
and rectify the warnings before releasing the software.

### Programmer's plateau ?

An Artistic impressionist painting of the programmer's plateau.

The programmer's plateau is when you make your objective to be fluent in a programming language. If you now safe file operations and link lists then that should be sufficient or know the programming language keywords. You can reach that plateau in a short space of time. Once you reach the plateau you are all dressed up with now where to go. Your objective should be not a programming language but framework. As an expert in a framework you are on your way to becoming a system architect. That is when you will be most beneficial to a business. A business who knows and understand the value of frameworks.

### Acknowledegments

I want to acknowledge my gratitude to those people who helped me to better myself so that I feel qualified enough to do a setup and a startup in fraemworks, paradigms of my choosing. Mr. Dimitri , 68K Assembly Language teacher, Ada Technical Lead A special thanks to the person who informed of the considerations I needed to know { connection pooling, load balancing, perfect database design } when I was transitioning from fat client to think client technology component technology. Also FAG-A-TUNA.

### Programming Fluency

During my TWO year computing course I was the best C programmer by far while other students never got the understanding. I also had diffulty in conceptualising The first language I ws taught was Pascal. Despite Pascal is meant to be an easy to learn for the purpose of learning programming. 68k assembly was my second language by this time the conceptualising a programming language was becoming natural . I was given a project to do in C. I was transitioning from accountancy to Computing because I found Accountancy boring and not something I wanted to do for the rest of my life. C programing was my third language and with the specification in my head from my accountancy career set me apart I couln't find a point to stop programming. I could go on for ever and forever. Other students did not have a specification so struggled with programming. You may find you have to stop yourself and say that is enough for now for an "A" Grade. As I did. It was during this project I realised I could do programming. Although when I started looking for a Job I was asked by employment agents how many thousands of lines of code I had written.

### Confusing Component Technology

\$ java org/makeuse/iSayHello
Error: Main method not found in class org.makeuse.iSayHello, please define the main method as:
public static void main(String[] args)
or a JavaFX application class must extend javafx.application.Application
Please note the lack of imagination and the aformentioned self inflicting confusion in the Naming convention with the variation on the word application Application. Complete failure in following any coding standards. The package name is a namespace. A way to uniquely indentify a class name. The java people have anticipated that the javafx library is going to be used for Application Development so just called the package application.Application. How silly ! What is the application developer suppose to call his prototype application.

### Importance of Naming Conventions

After the variable went out of scope it remained in memory then the second process (inner procedure) and the following processes picked the value from the same name variable and used it. By used it I mean in this case the value was concatenated. That is to say if variable A ="10" , variable B = "20" then if variable A is not initilaise at the beginning of a process or garbage collected at the end of a process and variable A is global Then the value of variable C {= A + B} will be "\$1020". But if the variable A is initialised or garbage collected then the value of variable C will be "\$20" which is correct. Variable B's content is of course overridden or reassigned by the incoming process with new data. SO the lack of initialisation for variable B has no effect.
With Use of Naming conventions If we rename those three variables where A is CountryCode , B is TelephoneNumber and C is TeleNumber. These three variables are part of a Telephone Compute Billing Engine. This Billing engine is run once a month to calculate customers' Telephone bills. If the customer made an international call on the 24th day of the month then if the CountryCode is not intialised then the customer would be over billed, every local after that call for that month would become an International call because CounrtyCode is appended to every local telephone number retrieve from the database. Customers can be over billed for years until some one made an international call at the beginning of the month and had a need to check their bill. The programmer confused himself with his own naming conventions and the testers had no hope of detecting the confused phD highly respected programmer. Perhaps the application of {MUM} with naming conventions such as localNumber InternationalNumber would not need half a line code of change of telephoneNumber = "" at the end of the routine. To unconfuse the bug. Avoid naming Variations when naming variables i.e. TeleNum ,TelNumber, TeleNumber , you satisfy the compiler but will end up confusing yourself.

#### Object-Orientated: Methodolgy or programming language ?

It may appear to be a silly question to a computer science professional, If you agree that Object orientated programming is a methodolgy therefore language neutral then please contact Department of Science Indiana University Bloomington https://cs.indiana.edu/ and let them know. The Indiana Science University is teaching RMI {object orientated}... unlike CORBA, which is intended to be language-neutral, What you give up in neutrality you make up in simplicity.

### Design Patterns

If a Class is a Blueprint and an Object is an Instance. The Instance is a Runtime Implementation of the Blueprint that is a Class. Then to add Impl to the class name the Class is not a Blueprint anymore, it then morphs into an Implementation when the Class Implements an Interface. Surely this is an illustration of Pindu-ism. The Oracle pindu has created a virus of confusion because he has relied heavily on Java compiler error message to support his supposation when he should have read a book on OOM which say you can define method body abstract classes but you can ONLY define < abstract > method declaration in interface class. This point is emphasize in the animal dog class illustration with comments on this one page website.

JAVA RMI: Secure by Design.
Once the client programme starts up, the application processing is executed in RAM memory by the kernel like any programme. The behaviour of the local application is transported on an as needed basis across the network over TCP/IP from server codebase somewhere to the client RAM memory. The transported object is garbage collected by JVM's Distributed garbage collection as soon as the client JVM releases reference to the object. The security is that there is no risk of some one decompilimg classes because there are no classes on client disk to decompile.

### Latency

The Indiana University suggests that Latency should beleft to the programmer. The programmer using RMI will be an application developer. It certainly isn't Application's developer's issue. I have also fixed the University's long standing Bug that Object-Orientated is NOT language neutral unlike CORBA. I have demonstrated this bug fix by showing Object-Orientated is a Methodology therefore language neutrality is NOT given up in favour of Simplicity.

JAVA RMI Proof of Concept: from UK TO Santa Barbara USA Distance: 18,000 km, Time Taken: Round Trip in 97 Milliseconds