Can the Integrated Test be linked to Software Requirements?

S

sarmila

#1
I am in the process of tracing the requirements to tests. I would like to know whether integration tests can be mapped to software requirements?

Please help.

thanks
Sarmila
 
Elsmar Forum Sponsor

Marc

Hunkered Down for the Duration with a Mask on...
Staff member
Admin
#2
Can you be a bit more specific, such as which tests, and what you mean by "integration tests"?
 

sagai

Quite Involved in Discussions
#3
Your post implies that there is a kind of reverse engineering going on. But may be I am mistaken.

So my coins would be, apart that more specific information would be helpful as mentioned already, that all tests could be linked to requirements.
But hang on a minute... let me rephrase ...
All requirements should be tested and for me it is rather difficult to imagine a test that does not origin from a requirement.
However if you are chasing requirements to a known set of tests, than I do not think your mission can end up with success, because this approach does not guarantee that you have all the requirements and all the corresponding tests. It only can end up with a conclusion that .. yes! ... all the tests we know traced back to requirements.

Let me share a food of thought ... When you have a set of tests that you consider as a comprehensive set of test of your software, than what is the meaning of having the requirement, other than to comply with a regulatory rigmarole?

Cheers!
 
Last edited:

glork98

Involved In Discussions
#4
I am in the process of tracing the requirements to tests. I would like to know whether integration tests can be mapped to software requirements? <snip>
You can map if you'd like, but it may not be needed. It depends on the overall process and, specifically, if you're mapping the code to the requirements. Then tie the Integration Tests to the code they test and from that to the requirements.

Perhaps you want the Integration Tests to do double duty and be Verification tests as well. That'd be OK if structured correctly and the product architecture accomodates that. The Verification should be against a full system. If you've got integrated items that are fully-functioning on their own then you can do it. I'm picturing that there are multiple programs operating simultaneously on one or more processors/cores/computers and the Integration test is for one of the programs.

If the Integration Test is covering a part of a larger executable image then I'd say "no" as far as making it a formal Verification Test. Showing that your Integration Tests, and Unit Tests, provide full coverage of requirements would be excellent. It'd be like showing SLOC coverage which can be near impossible in embedded environments.
 
S

sarmila

#5
Thanks for your replies.

The integration test covers the component/module integration testing and the system interface with external parties. For example, if the system has
battery, User Interface, Event log module, the integration test includes the test cases that check the integration between these modules.

Also if the system is communicating to an external device say a PC application, the test that checks the interaction between the system and PC application is also covered under the integration tests.

From the General Principles of Software Validation; Final Guidance for Industry and Staff document, "Software verification provides objective
evidence that the design outputs of a particular phase of the software development life cycle meet all of the specified reuqirments of that phase" I came to the conclusion that integration test cannot be linked to the System requirements.

However, some of the system requirements that cannot be easily tested through the system testing (black box) can be covered by the integration
testing. For example if a system requirement states that ,the system shall generate error on invalid user input, invalid date or time setting and calibration failure. This requirement is general one and not part of any official requirement doc.

The system test can easily cover the first two requirements, triggering the calibration failure in a system without invalid hardware component reading is not easy. One thought was to verify the third requirement in integration test and analysing the test report to check whether the requirement is met.

Is the above approach acceptable?
 
G

Gert Sorensen

#6
Verifying all requirements in software is always a tricky issue :)

My experience is that the easiest way of handling it (paperbased of course) is to have a traceability matrix that allows you to trace all requirements to verification, validation and integration tests. Some requirements may need to be tested in more than one way (e.g. both black box and integration tests), some may not. The good thing about using a matrix is that it ensures that all requirements do get tested, and it makes it easy to achieve compliance with design control requirements.

:bigwave:
 

glork98

Involved In Discussions
#7
However, some of the system requirements that cannot be easily tested through the system testing (black box) can be covered by the integration testing.
This is the heart of it. The system requirements, and system-level software requirements, should be tested against the system and not a sub-set of parts as is mandated by Integration tests.

At some point, ideal goals need to yield to practical considerations. If it's not practical or possible to do the testing in a full-up system then address the test approch in the test protocol and explain why the tests are necessarily done this way and why that's entirely appropriate.

In my current project we test 80%+ of the requirements in simulation. This is supported by the vast majority of the code being in a library that is linked with a small amount of code to enable the off-line simulation. The simulator for the microcontroller and hardware I/O components was validated with some basic testing for its purpose. The software being tested is not the entire image of the product but the binary for the vast majority and all the risk mitigation logic is the same between the simulation and the real product release.

I'm saying, if you need to do it, make the case and do solid work.
 

Peter Selvey

Staff member
Super Moderator
#8
According to IEC 62304, integration testing can be combined with system testing. Therefore, it is possible that integration test results can directly link to system requirements. For clarity it's best to state in the software development plan where integration tests are incorporated in system tests, then don't use term "integration tests" when tests are done on the whole system.

From experience, I think the reason IEC 62304 allows integration tests to be combined with system tests is simple efficiency: for a true non-system integration test, it is necessary to make up dummy input and output devices. Why not just use the real system? If so, the one test covers two purposes: the integration and system. I think this is actually done a lot in practice.

The approach works well for a simple system, where the data flows in a single line from one module to the next e.g. software that measures a sensor, does some signal processing, range checks, and then outputs the result to a display.

The approach does not work well for more complex systems that may have several modules interacting to give a result. In that case, leaving everything to system tests leads to a huge amount of test cases, so that integration testing might be more efficient to reduce the testing load.
 

glork98

Involved In Discussions
#9
Why not just use the real system?
Cost and schedule!

With an electro-mechanical real-time system it's incredibly hard and expensive to produce the inputs that test the boundary values of the requirements. In simulation the data is precisely valued and timed.
 
S

sarmila

#10
:thanx:

Thank you all once again.

Have decided to analyse the integration test results and link those to the system test and also to state the approach in the test plan.

Sarmila
 
Thread starter Similar threads Forum Replies Date
L Calibration Procedure for an IC (Integrated Circuit) Test Socket Calibration and Metrology Software and Hardware 6
lanley liao Does all of the suppliers need to integrated into the supplier list qualified of the company? Oil and Gas Industry Standards and Regulations 2
A 60601-1 : Integrated Dry ECG Electrodes = 2 Patient connections inside 1 applied part? IEC 60601 - Medical Electrical Equipment Safety Standards Series 5
A 60601-1-2 : Integrated Dry Electrodes = PATIENT COUPLED PORT? IEC 60601 - Medical Electrical Equipment Safety Standards Series 0
G Management Review (integrated system) Management Review Meetings and related Processes 17
J Audit Checklist for Integrated Management System for ISO 9001:2015, ISO 14001 & OHSAS18001 (IMS) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
G IMS (Integrated Management System) 9001, 14001 & 45001- Request Assistance ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
U Imagine a technology which is an artificial muscle and is integrated into textiles... Coffee Break and Water Cooler Discussions 1
M Interview question - IMS (Integrated Management Systems) a necessity or an ISO mistake? ISO 14001:2015 Specific Discussions 10
E Scales integrated in medical device IEC 60601 - Medical Electrical Equipment Safety Standards Series 4
B Efficacy of Q-Das system integrated with PC-DMIS Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
B Integrated management system - ISO 9001 and ISO 14001 - Is possible to have a joint certificate covering both standards? ISO 14001:2015 Specific Discussions 6
A Integrated risk and opportunities evalution template ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 0
T From certified QMS to certified integrated MS ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
A Integration of Information Security in an existent Integrated Management System IEC 27001 - Information Security Management Systems (ISMS) 4
B Integrated Quality Manual ISO 13485:2003 ISO 9001:2015 ISO 13485:2016 - Medical Device Quality Management Systems 6
G Integrated ISO EHSQ Multi-Site Certification ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
A Integrated Management Systems - ISO 50001 and ISO 9001 possible? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
M Integrated Phased Processes - AS9100D cl. 8.1 Operational Planning - Clarification AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
S Sample document for integrated ISO 20000 & ISO 27001 Other ISO and International Standards and European Regulations 3
W ISO 9001, ISO 14001 and 18001 Integrated Approach ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
T Guidance - Setting up an integrated quality & environmental management system Career and Occupation Discussions 12
G What to do and prepare for an Integrated Management Systems Audit ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
M IMS - Integrated ISO9001, ISO14001 & OHSAS 18001 Internal Audit Checklist Internal Auditing 3
S ISO 9001, ISO 17020 and ISO 3834 Joint (Integrated) QMS ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
S GDPMD integrated with ISO9001 Other Medical Device and Orthopedic Related Topics 3
D IMS (Integrated Management Systems) for 6 standards ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
D Integrated 9001, 14001, 18001 Management Review Meeting and Minutes Management Review Meetings and related Processes 10
B FMEA example for Cokemaking Process at a integrated Iron and Steelmaking Plant IATF 16949 - Automotive Quality Systems Standard 3
A Measureables to Monitor an Integrated Management System Capability IATF 16949 - Automotive Quality Systems Standard 3
G Integrated Systems Audit Mandays Duration - IAF Requirements ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
N Standalone Corrective Action Procedure or Integrated Nonconformance and Corrective Action 5
S Critique of an IMS (Integrated Management Systems) Manual Misc. Quality Assurance and Business Systems Related Topics 1
Claes Gefvenberg IMS (Integrated Management System) Audited for a week... Other ISO and International Standards and European Regulations 10
H Integrated Manual for ISO 9001:2008 and API Q1, Eighth Edition. Oil and Gas Industry Standards and Regulations 6
J Site Visit - Implementing an integrated management system ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
B How the indicator is integrated with other activities in the procedure? Customer Complaints 6
V Internal Audit of Integrated Management System Miscellaneous Environmental Standards and EMS Related Discussions 2
J Integrated Management System - Anyone with experience dealing with ISOQAR? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
L Open letter to inform our Vendors we are setting up an Integrated (QHSE) System Miscellaneous Environmental Standards and EMS Related Discussions 5
I Integrated Management System - Nonconforming Service/Product Defects ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
H ISO 26000 ... THE integrated management system? Various Other Specifications, Standards, and related Requirements 8
A Internal Audit for an Integrated Management System (IMS)? Internal Auditing 17
A Evaluating an Integrated Management System - Dissertation Topic ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
M Want Help to Implement IMS (Integrated Management System)? Occupational Health & Safety Management Standards 7
M How to Implement the QHSE Integrated Management System Easy Steps? Occupational Health & Safety Management Standards 1
E Integrated Standards Quality Manual Structured According to Clauses is better? Quality Manager and Management Related Issues 3
A Integrated Manuals that cover ISO 9001, ISO 14001 and OHSAS 18001 Quality Management System (QMS) Manuals 9
C 2nd Party Integrated Audits - PAS99 General Auditing Discussions 9
B Registration - Integrated TS 16949 and ISO 14000 IATF 16949 - Automotive Quality Systems Standard 8

Similar threads

Top Bottom