Get ready to learn what’s trending in software testing, how you can get ready to make the best out of these trends.
Currently, we witness enormous changes in technological advancements, specifically in terms of digitalization.
Top Software Testing Trends In 2023
Agile and DevOps
Organizations have embraced Agile as a response to the evolving conditions and DevOps as a solution. DevOps involves practices, rules, processes, and tools that help to integrate development, while reducing the time from development to the operations cycle.
DevOps has come as an extensively accepted result to reform software lifecycles right from development to delivery.
The relinquishment of both Agile and DevOps helps to develop and deliver quality software quickly, which in turn is also known as “Quality of Speed”.
This has gained interest over the recent years and continues to grow at an extensive pace.
Test Automation
The next software testing trend to note is test automation. In order to apply DevOps practices effectively, software developers cannot ignore test automation. It is a very important element in the DevOps process.
They need to find ways to replace manual testing with automated testing. As test automation is considered to be an important tailback of DevOps, retrogression testing should be automated.
And the fact that test automation is underutilized, with lower than 20% of testing being automated, there’s a lot of room to increase the relinquishment of test automation. Popular automation tools like Selenium and Katalon continue to evolve with new features to make automation much easier and more effective.
API and Services Test Automation
Segregating the client and server is the trend in Web and mobile design. API and services are reused in more than one operation.
These changes enable testing API and services independent from the operation using them. When API and services are used across customer operations and factors, testing them is more effective and effective.
The need for API and services test automation witness a steady increase. And having the right process, tool, and result for API automation tests is more critical than ever. Therefore it is crucial to learn and employ API Testing Tools for your testing systems.
Artificial Intelligence for Testing
Although applying artificial intelligence and machine learning to address the challenges in software testing isn’t new, the recent advancements in AI/ ML gives new openings to apply AI/ ML in testing.
Organizations will come up with strategies to optimize testing processes in AI/ ML. AI/ ML algorithms are developed to induce better test cases, test scripts, test data, and reports.
Smart analytics and visualization allows us to understand test content, areas of high threat,etc. We hope to see further operations of AI/ ML in addressing problems, test case prioritization, fault bracket, and assignment in the forthcoming times.
Mobile Test Automation
The trend of mobile app development continues to grow as mobile gadgets are more capable now than how they were before. To completely support DevOps, mobile test automation must be a part of DevOps toolchains.
The current application of mobile test automation as a software testing trend is veritably low due to the lack of tools. Using automated testing for mobile apps keeps increasing. This trend is driven by the need for more advanced tools for mobile test automation.
The integration between mobile device labs like Kobiton and test automation tools like Katalon may help in bringing mobile automation to the next level.
Test Environments and Data
The rapid-fire growth of the Internet of Things means further software systems are operating in multitudinous different surroundings.
This places a challenge on the testing engineers to ensure the right position of test content. Indeed, the lack of test environment and data is a top challenge when applying to test in agile projects.
We’ll see growth in cloud-based containerized test surroundings. The operation of AI/ ML to induce test data and the growth of data systems are some solutions for the lack of test data.
Integration of Tools and Activities
It’s hard to use any testing tool that isn’t integrated with the other tools for the operation lifecycle. Software teams must integrate the tools so that multi-source data can be gathered to apply AI/ ML approaches effectively.
For Example, using AI/ ML to describe where to concentrate testing on, needs not only data from the testing phase but also from the conditions, design, and implementation phases.
Along with the trends of DevOps we will see testing tools that allow integration with the other tools.
Conclusion
Watch out for these software testing trends in 2023 and keep track of all the exponential changes to know when to implement the right one for your needs.
Keeping up with these trends would give your testing team the advantage to stay ahead in the industry.
To know more about these trends and to know how to implement these the right way, you need to seek professional help.
Get in touch with our AlignMinds professionals who know the game so they can lead you through the process every step of the way in 2023.
There have been different software development methodologies in practice ever since developers started formalizing software development lifecycle to improve quality, efficiency and delivery time.
In a typical waterfall pipeline (the traditional software development model), the development lifecycle span across 6 stages.
Requirement analysis
Feasibility study
Product design
Software development/coding
Testing
Deployment
Even though the model added much-needed “professionalism” into the software development lifecycle, it was far from perfect due to a lack of agility and delay in deployment.
Since testing is performed during the final stages of the development lifecycle, any findings of errors, compatibility issues, vulnerability etc., resulted in a delay in delivery time as the code had to be sent back to the developers for correction.
Another drawback of the traditional model was that it caused wastage of resources. For example, consider that there is a flaw in the product architecture. Product architecture is decided in the early stages of the software development lifecycle. If the fault is identified in the earlier stage itself, the resources and time spent on designing and coding a product that is faulty from the beginning can be avoided.
Due to such flows in the waterfall model, the project heads were forced to find alternative strategies.
What is a shift left approach?
The shift-left approach aims to overcome the drawbacks of the waterfall model.
It advocates that testing should not be assigned to the last stages of the software development lifecycle; instead, it must be moved to the initial stages.
In other words, the shift left approach is nothing but a software development methodology in which testing is performed earlier in the lifecycle. The term “shift left” is used because testing is moved from the right side of the project timeline to the left side (Please refer to the image of the waterfall model).
Why shift left?
The Shift left approach tries to overcome the drawbacks of late testing.
It aims to identify faults in the requirement analysis, program architecture, design, and coding in the earlier stages itself so that wastages of resources and time can be avoided.
The model recognizes and promotes the significance of timely testing.
The model gives importance to the effective allocation of resources.
It advocates that enough resources should be allocated for testing.
The complexity involved in debugging is reduced. (Since quality and performance are checked in every stage of the development lifecycle, testing becomes a linear process rather than a one-time, resource-intensive task. Since every major change and integration are thoroughly scrutinized, it is easier to identify the elements that have an impact on quality, performance and compatibility).
Reduced code coverage and better encapsulation.
Projects are executed and products are released in a timely manner as there are fewer chances for any major reworks.
The quality of the product is increased due to the implementation of better approaches rather than easy solutions.
Due to the efficient use of time and resources, the overall cost of the project is reduced. Also, there are fewer chances for additional costs caused by reworks and late changes in resource allocation.
Since the product is released within the expected time, there are fewer chances for “missed opportunities”.
Since chances for a delay in the product release is very less, business lifecycle (Product planning, product development, product marketing and sales) and business strategies are not affected.
Shift left strategy
DevOps aims at adding velocity to the software development lifecycle without compromising on quality. In a way, both DevOps and Shift left approach aim for the same objectives. So, incorporating both into your product development strategy is a wise choice.
DevOps is a compound of development (Dev) and operation (Ops). Normally you can incorporate testing into the late (Shift right approach) or earlier stages (Shift left) of the development process.
While shift left strategy offer benefits such as better design, fast development time and delivery, better quality etc., better automation opportunities, wider and real-world testing, and better customer experience are the advantages offered by shift right strategy. To improve the effectiveness of both strategies, developers can adopt a common philosophy of “test early, test often and test in silos and production”. This will help the team to find flaws as early as possible, implement continuous testing and have a holistic approach.
Here are a few tips on how to plan and implement the shift left strategy.
How to plan shift left strategy?
When adopting a shift left approach, there are certain things that must be taken into consideration. We can categorize these factors into three categories mainly.
1. Budget
In a traditional model, the budget was not allocated evenly. A major chunk of the budget was spent on design and development alone. This led to apparent negligence of testing and quality assurance.
The shift left approach gives equal importance to each stage of the product development lifecycle. The budget is allocated more evenly, and testing is well taken care of. Here the terms budget and budgeting have a wider meaning. It involves a well-planned allocation of money and time so that every stage of the development lifecycle is strictly implemented.
2. Resource planning
Testing is a broad term. It involves different techniques and methods aimed at achieving different objectives. Examples of different testing techniques include manual testing and automated testing. Both techniques can be further divided based on their objectives such as load testing, unit testing, integration testing, functional testing, smoke testing, performance testing etc. While adopting the shift left approach, you must make sure that resources are allocated for every testing method and maximum testing techniques are implemented.
3. Test strategies
The test strategy should be well planned and documented. There should be a clear understanding of what testing techniques are going to be implemented at each stage of the lifecycle and which team will be responsible for the implementation. It is critical that the strategy is universally applied so that there is no wide swing in the quality of the product between each stage of the development lifecycle.
How to implement the shift left approach?
Plan it from the beginning
Once the requirement analysis is completed, the team should start planning on how to make the whole process shift left friendly. The plan should focus on incorporating testing as early as possible and implement it in a way that is incremental. The code coverage should be minimum, and encapsulation should be implemented while coding to make testing easier to execute. Also, resources should be allocated evenly across the development lifecycle.
Developers’ participation
Developers should test their own codes before pushing them to the main branch or QA team. Version control should be strictly implemented and any merging of codes should not affect the quality and performance of the main module. Since it is easier to test individual code modules, participation from developers ensures that the main branch is cleaner and error-free.
Testers’ participation
Following the same principle, testers should be introduced to common coding standards. They should be familiar with the functionality of the unit they are going to test. That way if they find any small mistakes in the code they are testing, they can correct the code themselves instead of sending it back to the developers. This will help to reduce development time and in fact, this is one of the main principles of DevOps. DevOps encourages developers to take participation in testing and testers to take participation in designing and development. They don’t have to be an expert in everything. However, expertise in one’s own field and a basic understanding of the whole development lifecycle are very much appreciated.
The QA team should also participate in any planning discussions. This will help them to understand the objectives of the projects better as well as the product design. This knowledge will help them to see whether it is the actual design that is being implemented into a final product. The presence of the QA teams in discussions will also help the other teams as testers can give a clear image of challenges that are most likely to emerge.
Early testing
The left shift strategy focuses on implementing testing from earlier stages of the development lifecycle. Instead of pushing testing to the final stages, it should be implemented from the unit level itself. This will help with avoiding common bugs and debugging complexities by keeping testing coverage to a minimum level. Also, since DevOps can heavily rely on automation, early testing is critical for proper implementation.
Readability and testability
DevOps emphasises team collaboration and multilevel participation. This is very much aligned with the principle of shift left approach that encourages testers to participate from the early stages of the development lifecycle. However, for the successful implementation of DevOps and shift left strategies, the dev team should follow certain principles such as code readability and testability. When codes are developed with a concern for their readability as well as suitability for testing, DevOps and shift left approaches become easier to implement. The developers can make use of element IDs, inline comments and notes, and incorporate aesthetics into their code to improve readability. Adopting the principles of encapsulation and lose coupling is one way to improve testability.
Best practices for shift left testing in DevOps
Planning and documentation
The whole development lifecycle should be well planned and documented. There should be participation from every team in the planning and discussions and the documentation should cover requirements analysis, project documentation, supporting documentation (Unit/feature level documentation with test cases) as well as inline documentation.
Universal quality standards
Since testing is pushed to the early stages and there are quality checks across the entire development lifecycle, the need for adopting universal quality standards is very crucial in the left shift approach. However, not every team and team member will be familiar with different quality standards. The QA team should help the other teams by first educating them on common quality standards. They can also create a document to outline various principles and parameters of quality assurance and provide a guideline on how to avoid common mistakes. The document should also help other team members to understand their share of responsibility in terms of testing and quality assurance.
Use Static Analysis
Static analysis or static code analysis is a method of debugging that is performed by examining the code without executing it. The basic code structure is examined thoroughly to ensure that it follows the best coding standards and practices. Static code analysis aims to find programming errors, syntax anomalies, substandard practices, and security issues in the code. Since the process can be tedious, it is better to automate static analysis before pushing it to the QA team. Since automation is involved, chances for false positives are high. So, the QA team should have a basic understanding of the code they are reviewing and should be familiar with various parameters of static code analysis. Proper inline documentation plays a vital role in static analysis.
Continuous testing and feedback
As stated earlier, both DevOps and shift left strategy encourage team collaboration and participation at multiple levels of the product development lifecycle. Program architects, designers, developers, testers and the operation team should continuously participate in the process and give feedback whenever necessary. The benefit of continuous testing and feedback is that the code modules will be cleaner and error-free when it reaches the next team involved in the lifecycle. As a result, the need for easy fixes and code reworks will be less to nil. Also here, the emphasis is put on shared responsibility.
Conclusion
Ever since the software industry embraced agile and DevOps principles, there was an overwhelming demand for ensuring quality in every stage of the software development lifecycle. The introduction of strategies such as MVP (Minimum Viable Product) and microservices etc. further illustrated the importance of executing testing at earlier stages of the development. The shift left approach is a result of various evolution and improvisation that happened in the software development industry.
Are you looking for pre-vetted DevOps engineers for your next projects? Contact our team of experts for a free consultation.
Today there are many test automation tools available in the market.
But, why do we need an automation tool for testing at all?
What makes use of automation tool a “must-have” in the Software development industry?
In today’s software world, demand for high-quality apps and products within very tight deadlines gives very less time for testers to perform quality testing. It is just impossible to test the complete application/product manually every time there is a bug fix or new releases.
Also, when it comes to testing scenarios like a large number of users access the website or an application simultaneously, it leaves no choice for an organization than to hire several hundred thousand testers (and test machines too) to do a manual test.
However, these challenges can be overcome by a tester or an organization if they choose an appropriate automation tool that fits their testing requirement. Choosing the right automation tool helps in saving resources, time, effort and cost. It can also help in effective testing delivering a high-quality product meeting the deadline and thereby increasing organization revenue and achieving customer satisfaction.
Katalon Studio is one such test automation tool that provides advanced test generation capabilities, faster execution and excellent reporting features. With the evolving features, leveraging in-built features and templates, Katalon is becoming tester’s favourite automation testing tool.
What is Katalon?
Katalon is a free-license automation solution for testing web and mobile application. It is developed by Katalon LLC. Katalon’s first release to the public was in September 2016.
Katalon Studio is built on the open-source automation framework Selenium and Appium. It is a very user-friendly and simple tool with an intuitive interface. Katalon is becoming the number one testing tool in the software testing field. While with other test automation tools, it takes weeks to automate test scripts, Katalon Studio requires just a few hours to achieve the same.
Why Katalon?
What makes Katalon so powerful and unique in comparison to other test automation tools?
The main advantage of Katalon over other automation tools is that it requires very little coding experience.
While several other test automation tools require testers to have technical skills to design test scripts, Katalon Studio’s specialized IDE and simple interface make it very user-friendly for testers who have no prior coding experience. Testers with some development knowledge can use the advanced features of this tool for the automation purpose. Even a black box tester can use this tool efficiently for the automated testing of web and mobile applications.
The time and efforts required to design test scripts are greatly reduced using this tool. Reusability/ scalability aspects are some other major features of Katalon Studio. Katalon makes use of several of Selenium features, you can still leverage the features in Katalon Studio to build your own scripts.
Highlights/Features of Katalon
Katalon Studio plays an important role in DevOps toolchain. Tests can be easily integrated with CI tools like Jenkins and TeamCity using Console Mode Execution or non-GUI mode in Katalon studio. One can make avail of command-line mode execution by installing the Katalon Studio plugins using Katalon Store’s API keys. This is a very powerful feature supported by Katalon.
Katalon Studio supports dual scripting interface. The black box testers with limited or no coding experience can use its simple interface to create test scripts. The tool has record and playback and manual mode for non-programmers. Katalon Recorder records the test actions, captures the web elements and plays the recorded steps. Testers can play their recorded scripts as many times as they want for their testing. The learning curve is very less as testers do not essentially need any coding experience in this case.
Testers with an advanced level of coding experience can use the scripting mode for test script creation. Those who have knowledge in Groovy can easily edit the test scripts in scripting mode. Katalon Studio allows reuse of these automated test scripts.
The tests recorded in Katalon Studio in one browser can be executed on multiple browsers (in parallel or sequential) speeding up the test cycle thus improving test coverage.
Using its well-structured framework and leveraging built-in features, Katalon Studio enables to provide high-quality applications. For example, Katalon supports parametrizing test objects dynamically which means users can control the object dynamically (sometimes the properties of an object might change due to business rules). The Test Objects can be handled in both manual modes and in script mode. Katalon provides systematic and neat UI with menus, tree views and tables so that one can organize well their test cases, objects and data files. This makes Katalon well-structured framework.
Features like Spy web utility is a great add-on in Katalon Studio. Using Spy web utility, object properties and locating method can be defined by the user themselves. This makes Katalon an intelligent tool.
While we must use other third-party libraries to generate reports in other automation tools, Katalon provides reports in graphical format. Also, it generates reports in HTML, CSV or PDF format.
Failures are reported with screenshots and logs.
Script creation time is very less when compared to other test automation tools.
Katalon Studio can be launched on Windows, Linux and Mac OS.
Cons of Katalon Studio
Katalon Studio supports only Apache Groovy for test script creation. However, it supports Java libraries and Java-compatible languages for test script execution.
Katalon is not open source, it has closed source code leaving no option to the community to customize or use community-built packages.
Katalon Studio does not support distributed testing currently.
Katalon Studio is more focused on functionality testing and does not support any load or performance testing.
Katalon Studio supports only web, mobile and API testing and does not support automating desktop application.
Sometimes the tool just slows down and freezes, mobile automation testing takes even more time to record scripts.
Choosing a test automation tool depends on the testing requirement of an organization. If the organization is looking for a tool to test the functionality of their web/mobile applications, Katalon would be the right decision. Katalon Studio mainly focusses on functionality testing, but with very little focus on testing of design aspects like screen size, orientation etc.
Katalon: The future
As the tool is new and in early stages in the market and the features are still evolving, it sometimes throws some bugs and just gets stuck during test script execution. More updates and bug fix releases are expected in future to make it more stable and smooth-running tool.
Also, support to more scripting languages for test script creation would be a great plus.
However, with all the great benefits provided by the tool and with its simple UI, the tool has already won the hearts of testing engineers. In spite of the drawbacks, Katalon Studio is becoming an emerging tool for test automation and is gaining popularity in the software development industry. Overall Katalon is an amazing tool and has a great future.
Jasmine is a behaviour-driven development framework, used for testing JavaScript code. To know about behaviour-driven development, we must know about test-driven development.
Test-driven development, as per definition, is a software development process that relies on the repetition of a very short development cycle: first, the developer writes an automated test case that defines a desired improvement or new function, then produces a minimum amount of code to pass that test, and finally refactors the new code to acceptable standards.
Behaviour-driven development, on the other hand, is a software development process that is emerged from test-driven development. The behaviour-driven development process can be called as a combination of general techniques and principles of test-driven development.
Why use Jasmine for testing?
Jasmine is an independent software as it does not depend on any other software development frameworks. It does not require a Document Object Model (DOM). A basic advantage that can be called of Jasmine is that its syntax is so obvious that it’s easy to understand. It also helps you to write your tests easily.
Working with Jasmine
Function
Let’s start with an example code that you want to test using Jasmine. We all know, in every programming language, we start with a Hello World program. Here also, let’s begin with a helloworld.js. A JavaScript function that returns “Hello World”.
Spec
Initially, you need to grab the latest standalone version of Jasmine on your computer. They are easily available on Google. All you need to do is search for it and download it. Unzip the downloaded file. The /src and /spec directories will have many files in it. You’ll have to empty them out as they are just examples which you probably won’t require.
Now, the function or let’s call the helloworld.js file should be put into the /src directory. We now have created the src. What we must do next is to create the spec.
The code has two parts.
The ‘describe’ part and the ‘it’ part. The ‘describe’ part will contain the main codes or functions that do the tests. ‘Describe’ is followed by the suite, which is just the English language and not any code, which helps to understand what it describes. Inside of ‘describe’, you have the ‘it’ part of the code. ‘It’ part is generally called as a ‘spec’.
The whole code might look like classes written in a function. ‘It’ describes what the code must do in general English language and in JavaScript code. You can have any number of specs in a suite.
Matchers
When a code is needed to be tested, you will definitely require a matcher. A matcher is something that will do the checking whether your code provides the required output. For beginners, we can use expect() and toEqual() as matchers. In our Hello World example program, we need the code to return the expected output – “Hello World”.
To test this, the matchers will run as expect (helloworld()).toEqual(“Hello World”); If that comes true, the program is successfully tested with no errors reported. There are many other matchers too. Matchers are selected according to the requirement of what is to be tested and what should be returned. In addition to this, you can make your own matchers too.
Example code
describe(“The ‘toEqual’ matcher”, function() {
it(“works for simple literals and variables”, function() {
var a = 12;
expect(a).toEqual(12);
});
it(“should work for objects”, function() {
var foo = {
a: 12,
b: 34
};
var bar = {
a: 12,
b: 34
};
expect(foo).toEqual(bar);
});
});
it(“The ‘toMatch’ matcher is for regular expressions”, function() {
var message = “foo bar baz”;
expect(message).toMatch(/bar/);
expect(message).toMatch(“bar”);
expect(message).not.toMatch(/quux/);
});
it(“The ‘toBeDefined’ matcher compares against `undefined`”, function() {
var a = {
foo: “foo”
};
expect(a.foo).toBeDefined();
expect(a.bar).not.toBeDefined();
});
it(“The `toBeUndefined` matcher compares against `undefined`”, function() {
var a = {
foo: “foo”
};
expect(a.foo).not.toBeUndefined();
expect(a.bar).toBeUndefined();
});
it(“The ‘toBeNull’ matcher compares against null”, function() {
var a = null;
var foo = “foo”;
expect(null).toBeNull();
expect(a).toBeNull();
expect(foo).not.toBeNull();
});
it(“The ‘toBeTruthy’ matcher is for boolean casting testing”, function() {
var a, foo = “foo”;
expect(foo).toBeTruthy();
expect(a).not.toBeTruthy();
});
it(“The ‘toBeFalsy’ matcher is for boolean casting testing”, function() {
var a, foo = “foo”;
expect(a).toBeFalsy();
expect(foo).not.toBeFalsy();
});
it(“The ‘toContain’ matcher is for finding an item in an Array”, function() {
var a = [“foo”, “bar”, “baz”];
expect(a).toContain(“bar”);
expect(a).not.toContain(“quux”);
});
it(“The ‘toBeLessThan’ matcher is for mathematical comparisons”, function() {
var pi = 3.1415926,
e = 2.78;
expect(e).toBeLessThan(pi);
expect(pi).not.toBeLessThan(e);
});
it(“The ‘toBeGreaterThan’ matcher is for mathematical comparisons”, function() {
var pi = 3.1415926,
e = 2.78;
expect(pi).toBeGreaterThan(e);
expect(e).not.toBeGreaterThan(pi);
});
it(“The ‘toBeCloseTo’ matcher is for precision math comparison”, function() {
var pi = 3.1415926,
e = 2.78;
expect(pi).not.toBeCloseTo(e, 2);
expect(pi).toBeCloseTo(e, 0);
});
it(“The ‘toThrow’ matcher is for testing if a function throws an exception”, function() {
var foo = function() {
return1 + 2;
};
var bar = function() {
return a + 1;
};
expect(foo).not.toThrow();
expect(bar).toThrow();
});
it(“The ‘toThrowError’ matcher is for testing a specific thrown exception”, function() {
var foo = function() {
thrownew TypeError(“foo bar baz”);
};
expect(foo).toThrowError(“foo bar baz”);
expect(foo).toThrowError(/bar/);
expect(foo).toThrowError(TypeError);
expect(foo).toThrowError(TypeError, “foo bar baz”);
});
});
Jasmine is independent. It does not depend on any other JavaScript frameworks.
It does not require a DOM.
It has a clean, obvious syntax.
Help maintainers understand the intention behind the code.
Brings validation and proper data handling concerns to the forefront.
Conclusion
There is plenty more you can do with Jasmine. Overall, Jasmine makes your testing fun. So, if you’re not yet into testing, now is an excellent time to start your JavaScript testing. It makes your testing pretty simple with Jasmine’s fast and simple syntax.
Mobile Application Market is gaining so much popularity now. With so many innovations happening in the field of mobile technology, mobile application testing is becoming a very essential part of the Software Development Life Cycle.
Excellent quality and highly efficient application have become a great challenge. To meet this challenge in a fruitful way, thorough and repeated testing of the system is required. Thorough testing can ensure that the application runs smoothly and meets user expectation. Effective test strategy and a combination of manual and automated testing tools are required to ensure a high-quality app.
Automated tests increase effectiveness and productivity and help maximum code coverage. Automated testing is sometimes considered as high cost, but if utilized effectively with balanced manual testing, the end result is overwhelming.
Why Test Automation?
There are many advantages to using Test Automation. It is fast and can be efficiently used for repeated testing. It can quickly find out the defects that occur due to the code changes thus, ensuring stability to the system. Repetition of tests can be effectively achieved using automated testing.
What is MonkeyTalk?
MonkeyTalk is one such automation tool used for efficient functionality testing of iOS and Android mobile applications. It is a cross-platform testing tool that record and playback functional test suites for iOS and Android applications running on real devices, emulators or simulator. Monkey Talk supports both native and hybrid iOS and Android apps.
MonkeyTalk has basically three components:
MonkeyTalk IDE – is an Eclipse-based tool that helps in the record, playback, manage, edit test suites for iOS and Android apps.
MonkeyTalk Agent – are the libraries that must be included in the mobile application under test. These Agents help the application to record and play MonkeyTalk commands.
MonekyTalk Scripts – describe different activities that need to be performed while testing.
Benefits of MonkeyTalk tool
A free, Open Source and simple to use tool.
Can be used for both iOS and Android applications.
Easy to understand and learn.
Testers need not be expert programmers to use the tool.
Support almost all iOS (iPhone, iPad, iPod etc.) and Android devices, either tethered or over the network.
Unlike other automation tools, MonkeyTalk uses Object-based recording. Object-based scripting is more flexible and unbreakable when compared to Image-based scripting.
Touch and gesture-based operations are easily captured by this tool. Actions like swipe, drag, move etc. can be efficiently recorded and played back.
MonkeyTalk is a cross-platform tool. For logically identical apps, MonkeyTalk commands recorded in one OS can be played in another OS.
Limitations of MonkeyTalk
Mobile app under test must be instrumented with MonkeyTalk Agent which means access to source code is required or help from the developer is needed. Also, 3rd party apps (like Camera, Map etc.) cannot be tested as the source code is not available.
Currently, it can be used only for iPhone and Android applications testing.
Conclusion
MonkeyTalk is a simple yet tremendously powerful test automation tool for iOS and Android applications. There are so many exciting things yet to happen in MonkeyTalk world and with all the upcoming features to test desktop apps, flex mobile apps, more reliable tests for game engines of mobile games etc., MonkeyTalk has a great future ahead of it!
In 1985, Canada’s Therac-25 radiation therapy machine malfunctioned due to software bug and delivered lethal radiation doses to patients, leaving 3 people dead and critically injuring 3 others.
During the first Gulf War, an American Patriot Missile system in Saudi Arabia failed to intercept an incoming Iraqi Scud missile due to a software rounding error in calculation. The missile destroyed an American Army barracks and 28 soldiers dead, 100 injured.
In May of 1996, a software bug caused the bank accounts of 823 customers of a major U.S. bank to be credited with 920 million US dollars.
Do you realize, how Testing is vital?
Do you think, understanding a product and testing the same against functionality, performance, security, GUI and many others is an easy task?
We can implement new & better choices for a product’s quality and security. There are lots of ways to test our product.
The most accepted deal is unit testing. Also, we should need to write functional or acceptance tests as well. For all these Codeception is the right choice! The Codeception testing framework figures out all these levels of testing and it is a multi-featured testing framework for PHP.
Codeception can handle unit, functional, and acceptance testing of web applications.
Major types of Tests
It was evoked in November 2011 and released the first stable version 1.0 in January 2012. Codeception tries to simplify and combine the process of writing tests, plugging different testing suites with the use of modules and it opens the way to anyone to extend and sharpen it. Codeception is testing framework in which all tests are written in a single descriptive manner.
What variety of tests & How?
Major types of Tests covered and its Pros & Cons
Acceptance Test (WebGuy)
Acceptance testing can be performed by a non-technical person. That person can be your tester, manager or even client. It allows us to test our applications using the normal website viewing process i.e.; visit a webpage, fill in a form, and submit the form to see the desired result.
The difference is with Codeception, we don’t have to waste time going to the browser each time we want to test a new feature out, instead, we can just run our acceptance tests to see its passes or not.
Try a sample scenario;
Probably the first test you would want to run would be signing in. In order to write such a test, we still require basic knowledge of PHP and HTML.
This scenario can probably be read by non-technical people. Codeception can even ‘naturalize’ this scenario, converting it into plain English:
It can be done by command:
The Want To section describes your scenario in brief. The $I object is used to write all interactions. The methods of the $I object are taken from the PhpBrowser and Db modules. We assume that all am commands should describe the starting environment. The amOnPage command sets the starting point of a test to the /login page. With the PhpBrowser you can click the links and fill the forms. That will probably be the majority of your actions.
Functional Test (TestGuy)
We can check our application without running it on a server, this is what done on Functional tests. These tests are written in the same way as Acceptance tests with PhpBrowser module enabled. It’s tested by a technically advanced guy i.e., TestGuy. The TestGuy knows how the application works, passes different $_GET, $_POST and $_REQUEST variables to assure the functionality. Codeception can connect to numerous web frameworks Symfony2, Laravel4, Yii2, Zend Framework and others which support functional testing.
We can open a web page with amOnPage command.
We can click links to open web pages of the application.
Functional tests will perform much better if we use powerful frameworks. It allows us to access and manipulate their internal states and this helps our tests shorter and faster. On the other hand, if we do not use frameworks there is no practical reason to write functional tests.
Let’s allow our application tested by this technically advanced guy for better results.
Unit Test (CodeGuy)
“Unit” casually refers to low-level tests and the developer understands how and what is tested here, though some would say a better name is DeveloperTest. The person testing, CodeGuy, knows the internals of the application and tests database operations and anything else that might need proof of concept. Codeception provides some well-built tools to make your unit tests simpler and cleaner. Even inexperienced developers should understand what is tested and how.
We can start with generating a classical PHPUnit test by this command:
We can use another command to create Codeception-powered unit tests.
Both tests will create a new ExampleTest file located in tests/unit directory.
A test created by generate:test command will look like this:
This class has predefined _before and _after methods to start with. We can use them to create a tested object before each test and destroy it afterwards. All Codeception tests are written in a descriptive manner. We can easily catch it from the test body. Its aim is to make tests easy to read, easy to write and easy to debug.
After all, this is how the Codeception works and you should give it a try on yourself.
In November 2013, Benedict Evans, a well-respected and widely followed analyst, said: “Mobile is Eating the World”. He foresaw the new revolution in the business and predicted it most accurately.
Mobile devices have witnessed a phenomenal growth in the past few years and on this Mobile-Era, people use mobile in Work, Gaming, Networking, Business, Education, Shopping and all day-to-day activities.
What about Mobile Apps?
As we all know, Mobile apps are software applications structured to run on Smartphone, Tablets, Computers and other mobile devices.
Every organization needs to ensure that the application meets a high-quality level in all circumstances. I think effective testing will help to achieve this eminent quality bar. Hence excellent testing is essential for addressing the challenges and complexities of a mobile application.
Mobile app testing: Elemental checklist
We’re seeing the current booming stage of Mobile apps; it is necessary to test apps carefully in all possible ways to reach a larger audience. Suppose if you are going to test a mobile app, we need to ensure our business goals, customer expectations and also follow up to the updated industry practices as well. The most important thing while you are testing a mobile app or even the web-based app is to make a checklist and test accordingly.
The major checklist categories are as follows:
Mobile App Checklist Categories
Device-specific checks
Installation, Uninstallation, Re-Installation and Updations: Verify whether the application can be installed, uninstalled, re-installed and updated successfully.
Verify that buttons/ keys which have no un-defined function & unexpected behaviour on the app when activating.
Verify the app behave as designed/desired if the sound on the device is turned off.
Verify whether the app behaves as designed/desired if the device is in aeroplane mode?
Verify that the app is found in the app store? (Check after go-live)
Verify whether the app switches properly to different applications on the device through multitasking.
Verify whether all touch screen positions (buttons) working when a screen protector is used.
Network-specific checks
Does the app behave according to specification if connected to the internet through Wi-Fi, 3G, and 2G?
Verify the behaviour of the application when there is a Network problem.
User should get a proper error message like “Network error. Please try after some time”
Resource Downloading: Verify whether the app handles the ‘Pause’ & ‘Resume’ to downloads.
Does the app resume work when it gets back into network reach from outside reach of the network?
Does the app use standard network ports (Mail: 25, 143, 465, 993 or 995 HTTP: 80 or 443 SFTP: 22) to connect to remote services, as some providers block certain ports?
App-specific checks
Has the app been tested on a different type of devices and different versions of OS?
Integration: Does the app connect correctly to the different social networks (LinkedIn, Twitter, Facebook, etc).
Is downloading of the app prevented in case it’s bigger than the OS allows downloading when connected to cellular networks?
The app does not interfere with other apps when in background/multitasking mode
Check if any payment gateway occurs like PayPal, Chargify, etc.
App UI checks
The main function of the app should be apparent immediately. It should speak for itself.
If there is a long list of data to scroll through, provide a search option above the list.
If performance is slow, indicate a progress status icon (“Loading…”), preferably with a specific message
Users should be warned of the consequences of deleting a document
All inactive buttons were clearly distinguished from active buttons
Check for navigation, tabs, page scrolling etc.
Store-specific checks
The app should not access information on the device outside the app without the user’s permission
The app cannot download code to be installed without the users’ consent.
The app can only get new functionality by way of an upgrade through the app store.
An app can’t be a “trail”, “beta”, “demo” or “test” version.
You cannot mention other app platforms in your app (for instance: “Also available on android!”)
Functionality should be in sync with the functionality described in store.
The app can’t use the user’s location without permission.
Interrupt Handling
Call
User should be able to accept/reject a call without any problems.
Sound of application should stop when the user is on the line.
The application should resume after user rejects/ends a Call.
SMS, MMS
User should get a Message alert while the application is running.
Messages should be received without any problems.
The application should resume after receiving Message.
Alarm Handling
Alarm should work without any problems while the application is running.
The application should resume after the Alarm is turned off.
Other elements of mobile app testing
Other app notifications should not affect app performance.
Storage low: Check the app by filling the memory and emptying it, and then compare the application performance.
Battery low: Check the app performance on low battery.
Having worked in various organizations for many years in the area of Software Testing, I could firmly say that an Agile based testing approach is becoming a critical factor in bringing superior quality to today’s web and mobile applications.
Though there are several testing approaches available, it is high time to come out of traditional approaches and adopt more rigorous and iterative testing practices that make testing more real and bring more effective results.
Agile testing is becoming very crucial while delivering quality results to the customer. I thought I will share with you some basic principles if followed that can bring drastic value additions to your test approach and make your client happy.
Agile Testing: Top 10 principles
Deliver value to the customer
Ensure that each prototype delivered to customer meet their expectation. Example: Is the layout of the website as expected by the customer, are the messages meaningful and user-friendly, does the click on button work, are there any broken pages etc.
Enable face-to-face communication
Direct communication with the client is very important to get a clear picture of the requirements. There might not be proper documents for the change requests as change occurs in a rapid manner in agile environment, however, if a representative from each team (developer, tester, program manager) meet together on a regular basis to discuss the amount of work done, it can help measure the work progress, determine bug status (how many severe bugs still exist, how many were re-opened etc.) and where we stand when compared to customer needs.
Keep it simple
Make a note of changes happening. Organized documents might not be available for every change that comes from the customer, but if you have a record of all changes (with date and comments) preserved at a commonplace, it will help you refer in future in case any confusions arise.
Also, when you report issues to the developer via bug tracking tool (or any other means like a simple spreadsheet), give a precise description of the issue that gives clear idea about the issue, if possible always attach a screenshot as the saying goes “A picture is better than 1000 words”.
Keep a big picture in mind
Although in agile testing methods, prototypes are delivered in short intervals, testers must not get distracted from the big picture of the overall product. Continuous change in code and implementation can sidetrack you from the original requirement of the client. Ensure that rigorous code change does not alter the output from the original agenda.
Provide continuous feedback
Testers must be proactive enough to provide timely feedback to developers and program managers about the quality (bugs founds, requirement missing etc.) of each prototype being delivered to client as each prototype is developed in short time spans and the early the feedback, the sooner the developers can fix those bugs, thus, ensuring delivery of quality product to the client. Focus more on people – make sure all teams (including developers, program managers, and testers) and the customer has the same understanding about a requirement.
Have courage
Sometimes a customer might not have an exact idea of what fits best for his/her requirement. Asking questions and offering ideas can help them choose the best fit for their requirement. Also, it gives you better insight into customer expectations.
Sometimes implementation must be questioned as it might not be the exact way customer is expecting. So, dare to question implementation and the way in which system is behaving as a developer who has developed that piece of the product might not have enough understanding about the requirement and your question can help them build the right piece of product.
It might create friction among teams as you cross-question their code; however, a friendly approach and the right attitude can lead to a fruitful discussion to build a product that fulfils customer needs. Ensure that your team is not intimidated by developers.
Picture by Dave Gray on Flickr
Practice continuous improvement
Always strive to improve your testing skills. Evaluate your testing skills at each delivery. How closely you tested customer requirements?
Were any of the areas left out while testing?
Did customer raise issues that you missed while testing?
Learn from your past experience, make a note of mistakes happened, list out how testing could have been improved at your side and try to implement new steps in your next project.
Respond to change
Change is a constant part of agile testing. Requirements change continuously, code change continuously. If you are having difficulty in keeping track of changes, make a practice of writing down each change (with date and comments that will help you remember why the changes were made etc.). Sometimes this kind of documentation is also helpful to other teams while re-visiting requirements or you can also show it to the customer when they say your product does not match their requirements.
Self-organize
Plan beforehand how your day is going to be. Be prepared to handle an unexpected situation. In an agile environment, there will be constant changes; so, keep a note of what tasks are to be tested on a daily basis and on what priority.
Keeping your work organized and a planned approach to handle your testing tasks can help you to test better and finish tasks in a more efficient and productive way.
Enjoy your work
Knowingly or unknowingly you are doing a very responsible part of bringing the world to a better place, so take this responsibility with great enthusiasm and passion. It gives a great sense of satisfaction to see the product you tested is the built-in right way that makes your customer happy, thus, bringing joyous results to your hard work.
