Can the Integrated Test be linked to Software Requirements?

S

sarmila

#11
glork98,

When the tests are conducted on the simulator, I believe the simulator must be validated, also the simulator should be developed following the guidelines for a Class A device. What are your thoughts on this?

Can you please let me know whether the validation process to test the simulator involved writing requirements, plans, test procedures and test reports?

thanks
Sarmila
 
Elsmar Forum Sponsor

Peter Selvey

Staff member
Super Moderator
#12
With an electro-mechanical real-time system it's incredibly hard and expensive to produce the inputs that test the boundary values of the requirements. In simulation the data is precisely valued and timed.
For boundary values that's fine, but basic functionality I've seen too many systems with software fail in third party testing to be confident about unit and integration tests.

The problems mostly fall into dynamic issues which become obvious when doing a whole system test, for example, seeing how a software based protection system reacts to a real hardware fault, or performance under unusual combinations of normal condition.

I think the background problem is that our brains are not wired for handling dynamic systems on paper, we need to physically see how the system responds to understand what is or is not important to test. Hence the tests at unit and integration levels often have limited effectiveness, which is again another reason to allow integration tests to be skipped.
 

glork98

Involved In Discussions
#13
glork98,

When the tests are conducted on the simulator, I believe the simulator must be validated, also the simulator should be developed following the guidelines for a Class A device. What are your thoughts on this?

Can you please let me know whether the validation process to test the simulator involved writing requirements, plans, test procedures and test reports?

thanks
Sarmila
Roughly, we were Class A. There are requirements and a validation plan. The testing was done both at a low level and a high level. Does a small set of instruction works? Does it work correctly when long sets of data are fed in?

So requirements, test protocol and test report were done.

It'd be a painfully hard thing to fully validate at the machine-language level. There are thousands of instruction permutations counting addressing modes. Then there's sequences of those. Yikes. We appraoched it a "validated to be suitable for the intended purpose." The MUC core simulator was shareware so we applied some SOUP principles.

We could have done more but once the Q&R staff was happy, we were happy.
 

glork98

Involved In Discussions
#14
For boundary values that's fine, but basic functionality I've seen too many systems with software fail in third party testing to be confident about unit and integration tests.

The problems mostly fall into dynamic issues which become obvious when doing a whole system test, for example, seeing how a software based protection system reacts to a real hardware fault, or performance under unusual combinations of normal condition.

I think the background problem is that our brains are not wired for handling dynamic systems on paper, we need to physically see how the system responds to understand what is or is not important to test. Hence the tests at unit and integration levels often have limited effectiveness, which is again another reason to allow integration tests to be skipped.
Excellent!

Simulation covers requirements well be are the requirements fully complete? No!

The simulation is used at three levels: unit, integration and functional parts of validation. The last is where inputs provoke clear actions. There is also a set of manual tests for UI functionality and we get good feedback from informal "monkey" testing. There are also unit tests run on the hardware to cover the low-level hardware drivers that can't be simulated.

The critical part of testing that can't be modelled are long sequences of user and device actions that are unplanned and include user "mistakes" combined with random device errors. The permutations tend to infinity. Good architecture and intentional design are the best tools to have a successful product.
 
Thread starter Similar threads Forum Replies Date
L Calibration Procedure for an IC (Integrated Circuit) Test Socket Calibration and Metrology Software and Hardware 6
lanley liao Does all of the suppliers need to integrated into the supplier list qualified of the company? Oil and Gas Industry Standards and Regulations 2
A 60601-1 : Integrated Dry ECG Electrodes = 2 Patient connections inside 1 applied part? IEC 60601 - Medical Electrical Equipment Safety Standards Series 5
A 60601-1-2 : Integrated Dry Electrodes = PATIENT COUPLED PORT? IEC 60601 - Medical Electrical Equipment Safety Standards Series 0
G Management Review (integrated system) Management Review Meetings and related Processes 17
J Audit Checklist for Integrated Management System for ISO 9001:2015, ISO 14001 & OHSAS18001 (IMS) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
G IMS (Integrated Management System) 9001, 14001 & 45001- Request Assistance ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
U Imagine a technology which is an artificial muscle and is integrated into textiles... Coffee Break and Water Cooler Discussions 1
M Interview question - IMS (Integrated Management Systems) a necessity or an ISO mistake? ISO 14001:2015 Specific Discussions 10
E Scales integrated in medical device IEC 60601 - Medical Electrical Equipment Safety Standards Series 4
B Efficacy of Q-Das system integrated with PC-DMIS Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
B Integrated management system - ISO 9001 and ISO 14001 - Is possible to have a joint certificate covering both standards? ISO 14001:2015 Specific Discussions 6
A Integrated risk and opportunities evalution template ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 0
T From certified QMS to certified integrated MS ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
A Integration of Information Security in an existent Integrated Management System IEC 27001 - Information Security Management Systems (ISMS) 4
B Integrated Quality Manual ISO 13485:2003 ISO 9001:2015 ISO 13485:2016 - Medical Device Quality Management Systems 6
G Integrated ISO EHSQ Multi-Site Certification ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
A Integrated Management Systems - ISO 50001 and ISO 9001 possible? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
M Integrated Phased Processes - AS9100D cl. 8.1 Operational Planning - Clarification AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
S Sample document for integrated ISO 20000 & ISO 27001 Other ISO and International Standards and European Regulations 3
W ISO 9001, ISO 14001 and 18001 Integrated Approach ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
T Guidance - Setting up an integrated quality & environmental management system Career and Occupation Discussions 12
G What to do and prepare for an Integrated Management Systems Audit ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
M IMS - Integrated ISO9001, ISO14001 & OHSAS 18001 Internal Audit Checklist Internal Auditing 3
S ISO 9001, ISO 17020 and ISO 3834 Joint (Integrated) QMS ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
S GDPMD integrated with ISO9001 Other Medical Device and Orthopedic Related Topics 3
D IMS (Integrated Management Systems) for 6 standards ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
D Integrated 9001, 14001, 18001 Management Review Meeting and Minutes Management Review Meetings and related Processes 10
B FMEA example for Cokemaking Process at a integrated Iron and Steelmaking Plant IATF 16949 - Automotive Quality Systems Standard 3
A Measureables to Monitor an Integrated Management System Capability IATF 16949 - Automotive Quality Systems Standard 3
G Integrated Systems Audit Mandays Duration - IAF Requirements ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
N Standalone Corrective Action Procedure or Integrated Nonconformance and Corrective Action 5
S Critique of an IMS (Integrated Management Systems) Manual Misc. Quality Assurance and Business Systems Related Topics 1
Claes Gefvenberg IMS (Integrated Management System) Audited for a week... Other ISO and International Standards and European Regulations 10
H Integrated Manual for ISO 9001:2008 and API Q1, Eighth Edition. Oil and Gas Industry Standards and Regulations 6
J Site Visit - Implementing an integrated management system ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
B How the indicator is integrated with other activities in the procedure? Customer Complaints 6
V Internal Audit of Integrated Management System Miscellaneous Environmental Standards and EMS Related Discussions 2
J Integrated Management System - Anyone with experience dealing with ISOQAR? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
L Open letter to inform our Vendors we are setting up an Integrated (QHSE) System Miscellaneous Environmental Standards and EMS Related Discussions 5
I Integrated Management System - Nonconforming Service/Product Defects ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
H ISO 26000 ... THE integrated management system? Various Other Specifications, Standards, and related Requirements 8
A Internal Audit for an Integrated Management System (IMS)? Internal Auditing 17
A Evaluating an Integrated Management System - Dissertation Topic ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
M Want Help to Implement IMS (Integrated Management System)? Occupational Health & Safety Management Standards 7
M How to Implement the QHSE Integrated Management System Easy Steps? Occupational Health & Safety Management Standards 1
E Integrated Standards Quality Manual Structured According to Clauses is better? Quality Manager and Management Related Issues 3
A Integrated Manuals that cover ISO 9001, ISO 14001 and OHSAS 18001 Quality Management System (QMS) Manuals 9
C 2nd Party Integrated Audits - PAS99 General Auditing Discussions 9
B Registration - Integrated TS 16949 and ISO 14000 IATF 16949 - Automotive Quality Systems Standard 8

Similar threads

Top Bottom