Advanced testing strategy for apps - Security Checklist 1

Agile methodologies and DevOps are widely adopted by many software service providers and consulting companies. The primary driver for this adoption being, faster delivery of products, independent teams, and generally better all-around synergy between the engineering and operations team.

In this DevOps and Agile world, the traditional modes of quality assurance, like manual testing, are not as effective when it comes to quality assurance. Granted, risk analysis, test planning, and test management are still important, but to ensure the same level of quality in an Agile world as before, or even better, organizations are using new skills such as test automation, data analytics, and AI technologies amongst others.

In this two-part blog(IInd part coming soon), we provide an example of such an advanced testing strategy via a checklist for mobile app testing on the two most popular platforms, iOS and Android. We cover a wide spectrum of categories ranging from tests related to how and where to store data, testing the entire authentication flow, code quality, platform interaction scenarios, and many more. You can download the entire checklist here, or for a detailed explanation, read on.

1) Data storage and privacy

Mobile devices are a constant companion in the digital era. They are used for various purposes such as entertainment, work, personal, professional, etc. Users interact a lot with mobile devices and invariably enter or store details about themselves. Contacts, bank account, health information, habits and preferences, travel logs, etc. Hence it’s no surprise that the number one priority while testing is data storage and privacy, i.e., to ensure that the app does not “leak” any confidential information.

For Android

1. No Sensitive information is stored in AndroidManifest.xml file
2. No Sensitive information is stored in Gradle.properties file
3. No Sensitive information is stored in any strings.xml folders in apk package
4. [Shared Preferences Check] Install apk on Android phone (or emulator) and navigate to /data/data/<package-name>/shared-prefs/keys.xml. Verify no sensitive data is being stored there.
5. [SQLite Databases Check] Install apk on Android phone (or emulator) and navigate to /data/data/<package-name>/databases/. Verify no sensitive information is stored in sqlite dbs
6. [Encrypted Database] Install apk on android phone (or emulator) and navigate to /data/data/<package-name>/databases and verify databases which have sensitive data are encrypted.
7. Verify that debug logs are disabled on production build. Connect device to your machine , run the following command adb logcat | grep “$(adb shell ps | grep <package-name> | awk ‘{print $2}’)” and verify the logs when app is running
8. Verify input fields that ask for sensitive data for e.g. “Password” are masked
9. Verify input fields that ask for sensitive data for e.g. “Password” does not display auto suggestions by default.
10. Verify input fields that ask for sensitive data for e.g. “Password” , Cut, Copy, Paste options should not work on these fields
11. By Default Backups should be disabled. In Androidmanifest.xml, verify android:allowBackup is set as false.
12. If Backup is a requirement, then check that no sensitive data is backed up.

Then run a backup from adb, adb backup -apk -nosystem <package-name>

ADB should respond now with “Now unlock your device and confirm the backup operation” and you should be asked on the Android phone for a password. Approve the backup from your device by selecting the Back up my data option. After the backup process is finished, the file .ab will be in your working directory.

Run the following command to convert the

.ab file to tar. dd if=mybackup.ab bs=24 skip=1|openssl zlib -d > mybackup.tar

Analyse the backup and check if there is any sensitive data stored.

For iOS

Verify no sensitive data Is stored in App Bundles DB. Run a simulator build, and navigate to

/Library/Developer/CoreSimulator/Devices/<Simulator ID>/var/mobile/Containers/Data/Application/$APP_ID/. read .db files

and verify no sensitive data is saved here.

Verify Keychain

/Library/Developer/CoreSimulator/Devices/<Simulator ID>/data/Library/Keychains/keychain-2-debug.db.

Data stored here should be encrypted

Logs should not have any sensitive data.

Verify input fields that ask for sensitive data for

e.g. “Password” are masked

Verify input fields that ask for sensitive data for

e.g. “Password” does not display auto suggestions by default.

Verify input fields that ask for sensitive data for

e.g. “Password” , Cut, Copy, Paste options should not work on these fields

2) Cryptography Requirements

While it’s important to test if data is stored securely, it’s equally important to verify if data is stored securely, i.e., is it encrypted. This can be defined as “How is data stored” compared to testing “Where is data stored.” Confidential information like passwords, secret questions, and answers, keys should never be stored in a human-readable format. Both Android and iOS use the AES 256 Keys algorithm to encrypt confidential information, and app developers must leverage this.

For Android

1. Verify sensitive data is encrypted when stored in the device. Encryption keys used should be saved in Android. Keystore

For iOS

1. Verify sensitive data is encrypted when stored in the device. Encryption keys used should be saved in Secure Keychain

3) Authentication and Session Management

Apps usually have a sign in, signup, and authentication mechanism. Authentication identifies a user. Depending on authentication, certain resources are authorized. A user logs in, consumes services, and eventually logs out. This is known as a session. The critical thing in this flow is to ensure that the user is correctly authenticated most safely, and only those resources are allocated to him for which he is authorized.

For Android

1. Passwords should have a strong Password Policy. Comprising of Minimum password length should be 8 characters. Password should contain the combination of following characters as mentioned below

a. Lower Case (a-z)

b. Numeric (0–9)

c. Upper Case (A-Z)

d. Non-Alphanumeric (e.g.!, @, etc.)

2. If needed 2FA Authentication should be present

3. When a password is entered multiple times, then app lockout should be implemented

4. Session IDs are always exchanged over secure connections

(e.g. HTTPS).

5. Verify The server verifies the session whenever a user tries to access privileged application elements,

(a session ID must be valid and must correspond to the proper authorization level).

6. Verify The session is terminated on the server side and session information deleted within the mobile app after it times out or the user logs out.

For iOS

1. Verify no sensitive data Is stored in App Bundles DB. Run a simulator build, and navigate to

/Library/Developer/CoreSimulator/Devices/<Simulator ID>/var/mobile/Containers/Data/Application/$APP_ID/. read .db files and verify no sensitive data is saved here.

2. Verify Keychain

/Library/Developer/CoreSimulator/Devices/<Simulator ID>/data/Library/Keychains/keychain-2-debug.db.

Data stored here should be encrypted

3. No sensitive data should be printed in Logs.

4. Verify input fields that ask for sensitive data for

e.g. “Password” are masked

5. Verify input fields that ask for sensitive data for

e.g. “Password” does not display auto suggestions by default.

6. Verify input fields that ask for sensitive data for

e.g. “Password” , Cut, Copy, Paste options should not work on these fields

Conclusion

This concludes the first part of this series, where we presented a checklist for testing mobile apps in the data storage & privacy category, cryptography requirements categories, and Authentication and session management category. In the next part, we will examine the test cases for the Network communications category, Platform Interaction, Code Quality and Build setting, and Resiliency category.

 

Scrum vs Kanban

Scrum is a project management framework based on the principles of agile methodology (https://agilemanifesto.org/) which emphasizes small production cycles over short sprints of 2–3 weeks. Since its an agile methodology It also stresses on feedback-driven incremental changes incorporated throughout.

Scrum is widely used in the software development process, but the principles of scrum are industry agnostic. It can be applied to any process or domain or business vertical.

“Agility is principally about mindset, not practices.”
― Jim Highsmith, Agile Project Management: Creating Innovative Products

Backlog

At the heart of a scrum is a project backlog. A project backlog is a well-authored list of tasks generally known as “Stories.” Stories are measured, or story pointed with some metric. E.g., 1,3,5,8,13 or 5 hours, 10 hours. As a rule of thumb, a project backlog should have sufficient tasks/stories to last at least two sprints. A business analyst or product owner prepares project backlogs in close collaboration with the customer during a periodic event known as Sprint planning. Representatives from the engineering, designing, quality are also involved in providing inputs on feasibility, cost, and time factors.

Sprints

Sprints are the actual production cycle or development cycle. Usually, sprints run for 2 weeks. At the start of a sprint, the project manager/ product owner includes stories or tasks from the project backlog. The development/engineering team works on the same for the duration of the sprint. Teams determine beforehand what constitutes the completion of a task. Usually, a task is complete once it’s tested and verified by the QA or customer. The end of a sprint features a new addition to the state of the existing product. At the end of the sprint, the primary stakeholder reviews the product and gives feedback. This feedback is used as an input for the next set of backlogs. This iterative cycle of

planning->develop->test->review->planning

forms the backbone of the scrum framework. A project manager often doubles as a scrum master who conducts the day-to-day sprint meetings; known as a Standup.

Standup

Standups are conducted on a day to day basis and are strictly time-bound. The goal of standup is for the team members to give a status update on a day to day basis. Members give an update on

“What they did yesterday?”

“What they will do today?”

“What are the challenges for the same if any.”

As the name suggests, standups are conducted standing up to put the focus on short time frames for providing updates

Retrospective

Scrum encourages a team to be self-motivated and drives continuous improvement. A “Sprint retrospective” is held to deliberate on the work accomplished in the previous sprint. The focus is on learning from past mistakes, improve efficiency, and share findings, which helps the team to improve and increase productivity.

Kanban

“In the beginning, it’s advisable to focus on the flow of your work and the idea that your work actually has a “shape.”
― Jim Benson, Personal Kanban: Mapping Work | Navigating Life

Kanban is a project management framework based on the principles of agile methodology with particular focus on the visualization of production flows. Kanban aims to improve productivity and reduce delays in the value chain by continuous monitoring, feedback and reduced waiting across the workflow.

Like its famous cousin “Scrum,” Kanban can be used across any domain or workflow. Any existing workflow can be expressed or represented using the Kanban board. There is no need to modify a workflow to include Kanban. An elementary Kanban board can be created with three lanes; “Todo,” “In progress,” “Done.” Depending on the workflow, these lanes can be more or less.

Visualization

Kanban places emphasis on visualization of workflows. On the kanban board, a workflow or its derivative is expressed in a naturally flowing order from left to right. For e.g., In the software development process, the workflow would be expressed with lanes as such

||Backlog|| — ||Progress|| — ||For review|| — ||For test|| -||For deploy||

Each task is expressed with cards, or color-coordinated. For e.g., In-progress tasks are expressed with green stickers or green colors. In-review tasks are expressed in Red color or red stickers. The tasks are moved to the adjacent lane, once it’s completed. Unlike a sprint, Kanban is not time-bound. Once a task is completed, it’s simply removed from the board.

By visualizing a workflow in such a manner, teams can obtain vital information about the work output. Useful metrics regarding the time taken to complete a task, where is a task getting held up, what is the dependency of one task over the other can be easily found out.

Kanban was first designed for the production lines of Toyota with emphasis on unused inventory. The Kanban board gives instant clarity to the team if a task is sitting idle on a lane for a long time. To overcome this, Kanban allows setting a limit on the input such that tasks are not lying idle in a lane because of the unavailability of any resource. This is generally known as W-I-P limit (Work in progress)

Typical benefits of Agile development methodology

1. A clearer view of process flow which helps in identifying bottlenecks
2. Flexibility in execution by incorporating smaller incremental goals as opposed to a monolithic, one-time development
3. Focus on continuous delivery by producing a subset of working software which is added to the existing product
4. Better tracking of issues and tasks
5. Continuous feedback leading to Increased productivity
6. Allows for change at any stage of the project development
7. Focus on the user all the time; their feedback & input

The primary goal of Kanban and Scrum is pretty similar, i.e., help organizations reduce execution time, optimize resource utilization, improve productivity & provide insights to measure. Both Kanban & Scrum are models of agile execution. Kanban focusses on specific aspects of the production process, whereas scrum is an entire framework for development. Kanban strives to present a clear picture of the workflow, while scrum defines process, steps & routines for workflow execution. It’s not uncommon to find Kanban being incorporated in a scrum process