Methods for determining software quality. Methods for assessing software properties. Ensuring reliability is the main motive for software development

Quality software(software) is determined based on a study of the external and internal features of the product. External quality is determined by how it works in real time, how productive it is for users. The second feature focuses on internal aspects that depend on the quality of the code written. The user focuses more on how the software works at the external level, the quality of which can only be maintained if the specialist has written good program code.

Software quality

Currently, there are two important approaches that are used to determine software quality:

1. Defect management.
2. Quality attribute.

Anything that does not meet the client's requirement falls into the category of defects. A design team that fails to fully understand the client's requirements will make design errors.

In defect management, defect categories are determined based on severity. The number of software problems is counted and action is taken according to the established severity. Control charts can be created to measure the capabilities of the development process.

Software quality has improved significantly over the past two decades. One reason for this is that companies are using new technologies such as object-oriented development and CASE tools. In addition, one can observe the growing importance of implementing management practices in production.

Software quality management is divided into three main areas:

1. Guarantee. Development of the foundations of organizational activities and software quality standards.
2. Planning. Selection of appropriate standards and adaptation to a specific software project.
3. Control. Define processes that ensure software development meets quality standards.

SQA Organization Policy

The organization's software quality policy must meet the following requirements:

• Compliance with the goals and objectives of the organization.
• Commitment to general quality assurance concepts.
• Commitment to quality standards adopted by the organization.
• Commitment to allocate adequate resources.
• Commitment to continuous improvement in the quality and productivity of the organization.

To ensure that all the requirements of the standard are met, companies appoint a quality officer. Responsibilities of this employee:

• Responsible for preparing the SQA's annual program of activities and budget.
• Organization of the development of plans for the development of the SQA system.
• Overall control over the implementation of the annual program of regular activities and planned development projects.
• Determining whether the program of activities matches the characteristics and scope of subcontractor services and software purchases planned for the coming year.
• Presentation and advocacy of SQA issues to executive management.
• Review proposals prepared by SQA for the annual program of events, checking the potential of the proposal to achieve its objectives.

High Level Concepts

Qualitative characteristics are high-level concepts that reflect important aspects and are not directly assessed for software quality. Instead, the plan should identify relevant indicators that can be used to determine one or more characteristics.

For example, when evaluating an XML parser, you can use the W3C XML Compliance Test Suite. It includes tests designed to satisfy all areas of control, as well as W3C Extensible Markup Language (XML) recommendations, with particular emphasis on the requirements for handling errors in the correctness or validity of XML documents. Thus, the percentage of test cases passed is used as a metric to evaluate the following characteristics of the XML parser in question:

• User perspective.
• Functionality.
• Reliability and fault tolerance.

From a user's point of view, there are several important characteristics that answer the following questions:

• Who provides the full range of necessary functions for their intended purpose?
• Does the software work reliably to produce the desired results when used correctly?
• Does the program function safely and reliably in the event of incorrect input?
• Is the software product easy to use?
• Does the software function quickly or does it seem unnecessarily slow?
• Does the program work well with another product the user is using?

Considering that quality issues are important to the user, the IT team responsible for deploying and maintaining the software may face other problems:

1. Protection against malicious attacks.
2. Quality of use of computing resources.

Low-quality resources are those that require more memory and processing power than necessary.

ISO provides this model with two new categories top level related to technological assurance of software quality.

ISO 9126 product requirements

ISO 9126 is an international standard for software evaluation. It is divided into four parts, covering the following topics:

• External indicators.
• Internal indicators.
• Quality model.
• Software quality indicators.

The first part of ISO 9126 is an extension of the previous standard carried out by McCall (1977), Boehm (1978) and FURPS in defining a set of quality characteristics.

• Functionality.
• Reliability.
• Usability.
• Maintainability.
• Portability.

Product functionality

Functionality is the main purpose of any product or service. The more uses a product has, the more difficult it becomes to determine its functionality. The software may have a list of what is available for it.

Some listed software characteristics (for example, usability) are present only to a certain extent, that is, they are not simply “on” or “off.” Many people confuse the overall functionality of a process and a software product. This is often because data flow diagrams (DFDs) and other modeling tools can reflect the functionality of a process as a set of transformed data into data out.

ISO 9126-1 and other quality models do not help measure the overall costs or benefits of a process, but only examine the software component. Relationship between functionality Software within an overall business process goes beyond the scope of ISO 9126.

The following attribute capabilities characterize the usefulness of the software in a given environment. Each of them can be measured only if the corresponding system programs are available.

Once a software system is operational, the reliability characteristic determines its ability to maintain the provision of its services under specified conditions for specified periods of time. One aspect of this characteristic is fault tolerance. For example, if the network goes down for 20 seconds, the system should be able to recover and continue operating.

The ability to learn to use a system (learning ability) is one of the main characteristics of usability.

Efficiency is related to the system resources used in providing the required functionality. Disk space, memory, and network are good indicators of efficiency. As with a number of other criteria, there is overlap between them. For example, the usability of a system affects its performance.

The ability to identify and correct a bug in a software component is what the maintainability characteristic refers to. Its indicators are influenced by the readability or complexity of the code, as well as modularity. This is what helps to identify the cause of the problem so that it can be eliminated.

Maintainability characteristics:

• Analyzability - identifies the root cause of failure.
• Variability - defines the effort that is put into modifying the code to eliminate a bug.
• Stability - demonstrates how stable the system is in operation when changes are made to it.
• Testability - determines how much effort goes into testing the system.
• Portability is the ability of a system to adapt to changes in its environment.
• Adaptability - How easily the system adapts to changes made to the specifications.
• Installation speed - how easily the system can be installed.
• Replaceability - How easily a system component can be replaced.
• Cost of software quality. She is very important. When a developer decides to do testing for their product, they are actually going to spend time, money, and effort to test it.
• Suitability - determines whether the software functions meet the requirements.
• Accuracy - establishes the correct implementation of functions.
• Interoperability - interacting with other system components.
• Software compliance with necessary laws and recommendations.
• Ensuring the quality and security of software and processing of data transactions.
• Reliability is the ability of software to operate under specified conditions for a specified period of time.
• Maturity - frequency of software failures.
• Recoverability is the idea of ​​a system’s ability to return to full operation after a failure.

Portability refers to how well software can adapt to changes in its environment or to its requirements. Object-oriented design and implementation techniques can contribute to the extent to which these software quality characteristics are present in a given system.

Cost of analysis processes

The cost of quality is calculated by analyzing the costs of conformance and nonconformity. The price of the first indicator is related to:

1. Prevention costs. This is the amount spent to ensure that all methods are followed correctly. It includes training teams, code reviews and any other activities related to QA.
2. Assessment costs. This is the amount of money spent on planning all the test tasks and then executing them, such as developing test cases.
3. Non-compliance costs. These are costs that arise due to internal and external failures.

Internal failures are costs that occur when test cases are executed internally for the first time, with some failing. Costs arise when a programmer must correct all defects identified in his piece during unit or component testing.

External failures are costs that occur when a defect is installed by the customer rather than the tester. These costs are much greater than those that appear internally. This is especially true if it intensifies software glitch.

Disciplined Process Analysis

This is a process assessment. It involves identifying and characterizing current practices, identifying strengths and weaknesses and the ability to control or avoid significant causes of poor product quality. Program audits can be of three types:

1. Self-esteem. Conducted within the organization's own personnel.
2. Third party assessment.
3. Third party assessment.

The software process audit is performed in an open general environment in order to improve its performance and using software quality assurance programs. The results of such an audit are confidential to the organization.

Regarding data collection, four methods are used:

1. Standard maturity checklist.
2. Individual and group interviews.
3. Document reviews.

Software development has reached such a level of development that there is a need to use engineering methods for assessing the results of its design at all stages of the life cycle, monitoring the degree of achievement of planned quality indicators and their metric analysis, assessing risk and the degree of use of ready-made components to reduce the cost of developing a new project. The basis of engineering methods in programming is improving software quality, to achieve which, methods for determining quality requirements, approaches to selecting and improving models for metric analysis of quality indicators and methods have been formed quantitative measurement quality indicators at all stages of the life cycle. Static techniques for assessing software quality are presented in Fig. 9.1. Dynamic techniques are somehow related to software testing.

Agreement on quality requirements, as well as clear communication to engineers of what constitutes quality in the resulting product, requires discussion and formal definition of many aspects. Engineers must understand the concept of quality, the characteristics, and the meaning of quality in relation to the software being developed or maintained. It should be noted that software requirements determine the required characteristics of software quality, and also influence the methods of quantitative assessment and the corresponding acceptance criteria formulated to evaluate these characteristics.

Software quality is a subject of standardization. According to GOST 2844-94 software quality is a set of properties (quality indicators) of software that ensure its ability to meet the needs of the customer in accordance with its purpose. This standard regulates the basic quality model and indicators, the main one of which is reliability. Standard 180/1EC12207 defines

Rice. 9.1.

divided not only the main processes of the software development life cycle, but also organizational and additional processes that regulate engineering, planning and software quality management.

According to this standard, the following software quality control must be carried out at all stages of the software development life cycle:

• checking compliance of requirements with the designed software product and criteria for their achievement;
• verification and certification (validation) of intermediate software results at the stages of life cycle and measurement of the degree of satisfaction with the achieved indicators;
• testing finished software, collecting data on failures, defects and other errors found in the system;
• selection of reliability models for assessing reliability based on the obtained test results (defects, failures, etc.);
• assessment of quality indicators specified in the requirements for software development.

Quality Inspection - This is a quality assurance process focused on the development team. It is used at all stages of software development.

Proof of correctness - it is a mathematical or logical technique used to convince oneself and others that a program does what it is supposed to do. Such a proof is a formal (rigorous) method.

For any engineering product, there are many interpretations of quality. Quality indicators may be required to varying degrees, may be absent, or may reflect certain requirements of the consumer and other interested parties, or be the result of a certain compromise (which is quite similar to the understanding of “acceptable quality,” a less rigid point of view on quality assurance as achieving excellence).

Cost of quality can be differentiated into the cost of preventing defects, the cost of assessment, the cost of internal as well as external failures. The driving force behind software projects is the desire to create software that has a certain value (meaningful for solving certain problems or achieving goals). The value of software can be expressed in the form of cost or some other form. The customer usually has his own idea of ​​the maximum cost investment, the return of which is expected if the main goals of creating the software are achieved. The customer may also have certain expectations regarding the quality of the software. Sometimes customers do not think about issues of quality and associated costs, so the discussion at this stage can be about the customer's full understanding of the costs and benefits associated with achieving a particular level of quality, and the degree to which the customer is involved in the decision-making process. Ideally, most of these decisions should be made during the requirements phase, but these issues can (and should) be raised throughout the software life cycle. There are no “standard” rules for exactly how these decisions should be made. However, engineers must be able to imagine different alternatives for achieving different levels of quality and their costs.

Software quality is a relative concept that only makes sense when taking into account the actual conditions of its application, and the requirements for quality must be related to these conditions and the specific area of ​​their application.

Software quality is characterized by three aspects: the quality of software, the quality of life cycle processes and the quality of maintenance or implementation (Fig. 9.2).

Quality Quality Quality

product support process

Rice. 9.2.

Aspect related to software life cycle processes, determines the degree of formalization and reliability of the software development life cycle processes themselves, as well as verification and validation (in short - V&V) of the intermediate results of these processes. Finding and eliminating errors in finished software is carried out using testing methods that reduce the number of errors and improve the quality of the product.

PP quality is achieved through the use of procedures for monitoring intermediate products at all stages of their life cycle, checking them to achieve required quality, as well as the use of software support methods. Effect of implementation the software largely depends on the knowledge of the operating personnel of the product functions and the rules for their implementation.

Software quality model has four levels of presentation.

First level corresponds to the definition of characteristics (indicators) of software quality, each of which reflects a separate user point of view on quality. According to existing standards (ISO/IEC9126, DSTU 2844-1994, DSTU 2850-1994, DSTU 3230-1995), the quality model includes six characteristics or six quality indicators(Fig. 9.3): functionality, reliability, usability, efficiency, maintainability, portability.

On second level define attributes for each specific quality characteristic that detail its different aspects. A set of these attributes is used to assess the quality of software.

Third level designed to measure quality using metrics, each of which, according to the 1SO/IEC9126 standard, is defined as a combination of a method of measuring an attribute and a scale for measuring its values. To assess quality attributes at the stages of software life cycle (when viewing documentation and programs, as well as program testing results), metrics with a given evaluation value are used.

Indicators-characteristics

Attributes

significant weight to level the results of metric analysis of the aggregate attributes of a specific indicator and quality as a whole. The quality attribute is determined using one or more assessment methods at the stages of the software life cycle and at the final stage of its development.

On fourth level To assess the quantitative or qualitative value of an individual attribute, an evaluative metric element, weight, is used. Depending on the purpose, features and conditions of software support, the most important quality characteristics and their attributes are selected (see Fig. 9.3).

The selected attributes and their priorities are reflected in the requirements for system development, or the corresponding priorities of the software class standard to which this software belongs are used.

Let's take a closer look software quality indicators.

Functionality. This is a set of properties that determine the software’s ability to perform a list of functions in a given environment in accordance with processing requirements and requirements for system-wide tools. A function is understood as a certain ordered sequence of actions to satisfy the consumer properties of the software. Functions can be targeted (main) and auxiliary. Here are the attributes that relate to functionality.

Functional completeness- a property of a software component that shows the degree to which the main functions are sufficient to solve problems in accordance with the purpose of the software.

Right (accuracy)- an attribute that shows the degree to which correct results are achieved.

Interoperability - an attribute that shows the ability of software components to interact on special systems and environments (OS, network, etc.).

Security - an attribute that determines the software's ability to prevent unauthorized access (accidental or intentional) to programs and data.

Reliability. This is a set of attributes that determine the software's ability to transform input data into results under conditions that depend on the software's lifetime (wear and tear and aging are not taken into account). Reduced software reliability occurs due to errors in requirements, design and implementation. Failures and errors in programs appear over a given period of time.

The subcharacteristics (subcharacteristics) of software reliability include the following.

Reliability - an attribute that determines the ability of software to function without failure (both program and hardware).

Error tolerance- an attribute that shows the software’s ability to perform functions under abnormal conditions (hardware failure, errors in data and interfaces, violations in operator actions, etc.).

Recoverability - an attribute that indicates the software's ability to restart to re-execute and recover data after failures.

Some types of “critical systems” (real time, radar, security systems, communications, etc.) are subject to requirements to ensure high reliability (error tolerance, accuracy, reliability, ease of use, etc.). Software reliability largely depends on the number of remaining and unresolved errors during its development at the life cycle stages. During operation, errors are detected and eliminated. If, when correcting errors, no new errors are introduced, or at least fewer new errors are introduced than are eliminated, then during operation the reliability of the software continuously increases. The more intensively the operation is carried out, the more intensively errors are identified and the faster the software reliability increases.

The following factors affect software reliability:

• a set of threats leading to adverse consequences and damage to the system or its operating environment;
• threat as a manifestation of a violation of system security;
• integrity as the ability of a system to maintain stable operation and not have risk.

Detected errors may be the result of external threats or failures, they increase risk and reduce some properties of the system's reliability.

Reliability is one of the key problems of modern software systems, and its role will constantly increase as quality requirements constantly increase computer systems. New direction - software reliability engineering(Software reliability engineering) - focused on the quantitative study of the operational behavior of system components in relation to the user expecting reliable operation of the system, and includes the following aspects:

reliability measurement, i.e. carrying out its quantitative assessment using predictions, collecting data on the behavior of the system

during operation, as well as modern reliability models;

• strategies and metrics design and selection of finished components, the process of developing a component system, as well as the operating environment that affects the reliability of the system;
• application of modern inspection methods, verification, validation and testing during the development of the system, as well as during its operation.

Verification used to determine the compliance of finished software with established specifications, and validation - to establish compliance of the system with the user requirements that were presented by the customer.

In reliability engineering term suitability(dependability) refers to the ability of a system to have properties desired by the user, who is confident in the high-quality performance of the functions specified in the requirements. This term is defined by an additional number of attributes that the system must have:

• availability;
• readiness for continuous operation (reliability);
• safety for environment, i.e. the ability of the system not to cause catastrophic consequences in case of failure (safety);
• secrecy and safety of information (confidential);
• the ability to preserve the system and resistance to its spontaneous change (integrity);
• ability to operate the software, ease of performing maintenance operations, as well as eliminating errors, restoring the system after they are eliminated (maintainability);
• readiness and safety of information (security), etc. Achieving system reliability is ensured preventing failure(fault prevention) or its eliminating(removal fault), as well as assessing the possibility of new failures and measures to combat them.

For numerical assessment of reliability, methods of probability theory are used. Each software component, its operations and data are processed at discrete times, for example 5, 28,..., pb.

Let it take time T after the first unsuccessfully processed system component, a failure appeared for which the following expression is valid:

P( T > nb)= (1 - L.) And, where L. is the probability of failure, with the average

the waiting time will be equal T = -.

Let's assume that the value 5 decreases and the time T remains fixed, then we have

R(T>t) =

Where t- time to failure, in this case, is a continuous value distributed exponentially.

Thus, ensuring software reliability is a labor-intensive process that requires creating stable operation of the system with respect to software failures, i.e. ensuring a sufficiently high probability that the system will recover spontaneously at some point after a failure occurs in it.

Ease of use. This indicator is characterized by many attributes that determine the necessary suitable conditions for using the software (for example, interactive, non-dialogue) by a given range of users to obtain appropriate results. In the DSTU 2850-1994 standard, ease of use is defined as a set of software attributes that characterize its ergonomics:

• understandability - an attribute that determines the effort expended on recognizing logical concepts and conditions of application of the software;
• studyability(ease of learning) - an attribute that determines the efforts of users spent on determining the applicability of the software through the use of operational controls, diagnostics, as well as established procedures, rules set out in the documentation;
• efficiency - an attribute that determines the system's response when performing operations and operational controls;
• consistency - an attribute that shows the development's compliance with standards, agreements, rules, laws and regulations.

Efficiency. This is a set of attributes that determine the relationship between levels of software execution, the degree of use of resources (facilities, equipment, materials - paper for a printing device, etc.) and services performed by full-time maintenance personnel. Software performance attributes include:

• reactivity - an attribute that shows response, processing and execution times of functions;
• resource efficiency - an attribute indicating the amount and duration of resources used when performing software functions;
• consistency - an attribute that shows compliance of a given attribute with specified standards, rules and regulations. Maintainability. These are a set of properties that determine the effort that must be expended to carry out modifications, including adjustments, improvements and adaptations of software when the environment, requirements or functional specifications change. Maintainability is determined by the following attributes:
• analyzability - an attribute that specifies the effort required to diagnose failures or identify parts that will be modified;
• changeability - an attribute that determines the possibility of removing errors in the software or making changes to eliminate them, as well as introducing new features into the software or operating environment;
• stability - an attribute indicating the constancy of the structure and the risk of its modification;
• testability - an attribute that determines the efforts during software validation and verification in order to detect non-compliance with requirements, as well as the need for software modification and certification;
• consistency - an attribute that indicates the attribute's compliance with the conventions, rules, and regulations of the standard. Portability. This is a set of indicators that determine the software's ability to adapt to work in new conditions of the runtime environment. The environment can be organizational, hardware and software. Transferring software to a new execution environment may be associated with a certain set of actions aimed at ensuring its functioning in an environment different from the one in which it was created, taking into account new software, organizational and technical capabilities. Portability includes the following attributes:
• adaptability- an attribute that determines the effort spent on adapting to different environments;
• customizability(ease of installation) - an attribute that determines the required effort to run this software in a special environment;
• coexistence- an attribute that determines the possibility of using special software in the operating system environment;
• replaceability- an attribute that provides the possibility of interoperability when working together with other programs with the necessary installation or adaptation of software;
• consistency - an attribute that determines compliance with software portability standards or conventions.

Software quality is the degree to which the software has the required combination of properties.

Software quality is the collection of software characteristics related to its ability to satisfy stated and intended needs.

Software quality characteristics

Functionality(Functionality) - determined by the ability of the software to solve problems that correspond to the recorded and expected needs of the user, under given conditions for using the software. Those. This characteristic ensures that the software operates properly and accurately, is interoperable, meets industry standards, and is protected from unauthorized access.

Reliability Reliability – the ability of software to perform required tasks under specified conditions over a specified period of time or a specified number of operations. The attributes of this characteristic are the completeness and integrity of the entire system, the ability to independently and correctly recover from failures, and fault tolerance.

Ease of use(Usability) - the ability to easily understand, study, use and attractiveness of the software for the user.

Efficiency(Efficiency) – the ability of the software to provide the required level of performance in accordance with the allocated resources, time and other specified conditions.

Ease of maintenance(Maintainability) - the ease with which software can be analyzed, tested, changed to correct defects, to implement new requirements, to facilitate further maintenance and adapt to the given environment.

Portability(Portability) – characterizes software in terms of the ease of its transfer from one environment (software/hardware) to another.

Software Quality Model

On this moment most common and used multi-level software quality model, presented in a set of standards ISO 9126. On upper level highlighted 6 main characteristics of software quality, each of which is defined by a set of attributes that have corresponding metrics for subsequent evaluation ( see fig. 1).

Fig. 1 – Software quality model (ISO 9126-1)

29. Software quality characteristics (GOST).

GOST ISO 9126-93

The quality of software can be assessed by the following characteristics:

4.1 Functionality(Functionality) A set of attributes related to the essence of a set of functions and their specific properties. Functions are those that fulfill stated or anticipated needs: Notes

1 This set of attributes characterizes what the software does to satisfy a need, while other sets primarily characterize when and how it is performed.

4.2 Reliability Reliability A set of attributes related to the ability of software to maintain its level of performance under specified conditions over a specified period of time. Notes 1 There is no wear or aging of the software. Reliability limitations occur due to errors in requirements, design, and implementation. Failures due to these errors depend on how the software is used and the software versions previously selected. NOTE 2 ISO 8402 defines “reliability” as “the ability of an element to perform a required function.” In this standard, functionality is only one of the characteristics of software quality. Therefore, the definition of reliability is expanded to “maintain its level of performance” instead of “performing a required function” (see also 3.4).

4.3 Practicality(Usability) A set of attributes related to the scope of work required for use and the individual assessment of such use by a specified or intended range of users. Notes 1 “Users” can be interpreted as the majority of direct users of the interactive software. Users may include operators, end users and indirect users who

affected by this software or that depend on its use. Practicality must be considered across the variety of user operating conditions that may affect the software, including preparation for use and evaluation of results.

2 Usability, defined in this standard as a specific set of attributes of a software product, differs from the definition from an ergonomic point of view, where other characteristics such as efficiency and inefficiency are considered as components of usability.

4.4 Efficiency(Efficiencies) A ​​set of attributes related to the relationship between the level of quality of software operation and the amount of resources used under specified conditions. Note - Resources may include other software products, technical means, materials (for example, printing paper, floppy disks) and services of operating, maintenance or maintenance personnel.

4.5 Maintainability(Maintainability) A set of attributes related to the amount of work required to carry out specific changes (modifications). NOTE Change may include corrections, enhancements or adaptation of the software to changes in the environment, requirements and operating conditions.

4.6 Mobility(Portability) A set of attributes relating to the ability of software to be ported from one environment to another. NOTE The environment may include the organizational, technical or software environment.

Every day in our work we come across the rather abstract concept of “software quality” and if you ask a tester or programmer a question – “what is quality?”, then everyone will have their own interpretation. Let's consider the definition of "software quality" in the context of international standards:

Software quality is the degree to which the software has the required combination of properties.

Software quality is the collection of software characteristics related to its ability to satisfy stated and intended needs.

Software quality characteristics

Functionality(Functionality) - determined by the ability of the software to solve problems that correspond to the recorded and expected needs of the user, under given conditions for using the software. Those. This characteristic ensures that the software operates properly and accurately, is interoperable, meets industry standards, and is protected from unauthorized access.

Reliability Reliability – the ability of software to perform required tasks under specified conditions over a specified period of time or a specified number of operations. The attributes of this characteristic are the completeness and integrity of the entire system, the ability to independently and correctly recover from failures, and fault tolerance.

Ease of use(Usability) - the ability to easily understand, study, use and attractiveness of the software for the user.

Efficiency(Efficiency) – the ability of the software to provide the required level of performance in accordance with the allocated resources, time and other specified conditions.

Ease of maintenance(Maintainability) - the ease with which software can be analyzed, tested, changed to correct defects, to implement new requirements, to facilitate further maintenance and adapt to the existing environment.

Portability(Portability) – characterizes software in terms of the ease of its transfer from one environment (software/hardware) to another.

Software Quality Model

Currently the most common and used multi-level software quality model, presented in a set of standards ISO 9126. Highlighted at the top level 6 main characteristics of software quality, each of which is defined by a set of attributes that have corresponding metrics for subsequent evaluation (

1

The work provides brief information according to ISO 9126 standard " Information technology. Software product evaluation. Quality characteristics and guidelines for their use." Since the range of software systems is wide, a group of systems was selected to create tests. To assess the quality of this group of programs, the following is given: short description some software packages. Based on the characteristics of this group software, a list of characteristics that determine the quality of programs was compiled. According to recommendations ISO standard 9126, a study was conducted of the characteristics of a selected group of software tools. Registration of characteristics (yes or not) and expert assessment were used as a method for determining the values ​​of quality indicators. Thus, a methodology has been proposed for assessing the quality of one type of software in accordance with the ISO 9126 standard.

standard

characteristic

Quality control

software

1. Baranyuk V.V., Tyutyunnikov N.N. Assessing the quality of electronic dictionaries and encyclopedias // Software engineering. – 2012. – No. 8. – P. 29–37.

2. Glichev A.V., Panov V.P., Azgaldov G.G. What is quality? – M.: Economics, 1968. 135 p.

3. Gorbachenko I.M. Software for creating automated teaching systems // Problems of regional informatization. PIR-2005: materials of the ninth scientific-practical conference(Krasnoyarsk, October 11–12, 2005). – Krasnoyarsk: IPC KSTU, 2005. – vol. 2. – pp. 132–135.

4. Gorbachenko I.M. Comparative analysis existing systems testing // Testing in the field of education: problems and development prospects: materials of the All-Russian scientific and practical conference. (Krasnoyarsk, May 19-21, 2008) / rep. ed. G.P. Karlov. – Krasnoyarsk: SibSTU, 2008. – P. 177–183.

5. Lipaev V.V. Problems of quality assurance of complex software [ Electronic resource]. – Access mode: http://quality.eup.ru/MATERIALY4/poksps.htm (date accessed 04/9/2013).

6. Lozinin A.I., Shubinsky I.B. Characteristics of software quality and methods for their evaluation [Electronic resource]. – Access mode: http://www.ibtrans.ru/Estimating %20methods.pdf (access date 03/12/2013).

On modern computers A lot of different software has been installed. And I want it to be of high quality, efficient, work without failures, etc. Let's consider the definition of “Software Quality” in the context of international standards:

1) Software quality is the degree to which the software has the required combination of properties. ;

2) quality of a software tool - a set of properties of a software tool (SS), which determine its suitability to satisfy specified or implied needs in accordance with its purpose [GOST 28806-90 “Quality of software tools. Terms and Definitions"].

The purpose of this work is to develop a methodology for applying the requirements of the ISO 9126 standard to assessing the quality of one of the types of software - test creation systems.

ISO 9126 standard

At the moment, the most widespread and used is the multi-level software quality model presented in the ISO 9126 set of standards. The basis for regulating system quality indicators is international standard ISO 9126 “Information technology. Software product evaluation. Quality characteristics and guidelines for their use." This standard describes the multi-level distribution of software characteristics. At the top level, 6 main characteristics of software quality are identified, each of which is determined by a set of attributes that have corresponding metrics for subsequent evaluation (Figure).

According to this model, the functionality of a software tool (functionality) is a set of software properties, determined by the presence and specific features of a set of functions capable of satisfying specified or implied quality needs along with its reliability as a technical system. Reliability is the ability of software to perform required tasks under specified conditions over a specified period of time or a specified number of operations. Ease of use of a software tool (usability) is a set of software properties that characterizes the efforts required to use it and the assessment of the results of its use by a given circle of software users. Efficiency is the ability of the software to provide the required level of performance in accordance with the allocated resources, time and other specified conditions. Maintainability is the ease with which software can be analyzed, tested, changed to correct defects, to implement new requirements, to facilitate further maintenance and adapt to the given environment. Portability is a set of properties of a software system that characterizes its adaptability for transfer from one operating environment to another.

Software Quality Model (ISO 9126)

Software for creating test systems

At modern level With the development of computer technology and information exchange systems, testing is increasingly used in teaching, which is used as a tool for university monitoring and forecasting. Monitoring as a control and diagnostic system provides the teacher with objective and timely information about the level of students’ mastery of compulsory educational material, and the administration about the effectiveness of management. The computer testing system is a universal tool for determining students' learning at all levels of the educational process.

Creating tests at a high methodological level requires the teacher to develop a clear conceptual and terminological structure of the course, i.e. tables of concepts and theses tested in tests, structured by topics and sections of the curriculum of the academic discipline.

The computer testing system is an integral component for the future development of distance learning.

Nowadays, ready-made tools for developing training programs are increasingly appearing. Moreover, these developments are not only foreign (for example, Adobe Acrobat, Macromedia Authorware, ToolBook II, Quest and others), but also domestic ones (for example, HyperMethod, “Associate Professor”, “Prometheus”, “OROX” network shell, CADIS). Let's give brief description some of them.

One of the testing systems “Test Designer” is a universal knowledge testing system (system website - http://www.keepsoft.ru/simulator.htm). The program supports five types of questions: closed (for choosing one or more answers), open (entering an answer), matching and ordering. This allows you to carry out any tests. The tests have the ability to use music, sounds, images and videos. Any data can be printed on a printer. Several people can independently take tests on one computer, logging into the program under their own names.

Next package - system testing INDIGO(website - http://indigotech.ru/). In this system you can also create 5 types of test tasks. But besides this, a feature of the INDIGO test designer is its support for multi-level hierarchical grouping of test questions by tasks, topics, etc. After all, if test questions are displayed in one linear list, then difficulties arise with navigation and understanding of which question relates to what. In this system, it is possible to set individual settings for each group (in particular, the order of issuing nested elements or their random selection).

The next package under consideration is VeralTest - a set of programs for creating test tests and for organizing multi-user computer testing (website - http://veralsoft.com/veraltest.shtml). In this system, the following types of back-end tests can be created - closed (selecting one answer and selecting multiple answers), entering a text answer, entering a numeric answer, matching questions.

The VeralTest software package is presented in two editions:

VeralTest Express. Allows you to create stand-alone self-running tests (exe tests), which can be run on any computer without prior installation and configuration. The VeraTest Express package includes: TestEditor test editor and ResultViewer program for viewing test results.

VeralTest Professional. Supports all features of the express edition. In addition, the package includes a testing server (TestServer program), which allows you to organize testing in a computer class or local network enterprises. In this case, access to the tests is carried out through a web browser (for example, Internet Explorer, Google Chrome, Mozila Firefox). This edition also includes the TestAdmin administration program, with which you can register users, combine them into groups, assign tests for users to perform, view and print test results.

In addition to the above software systems, there are many other systems. Some of them can be found on the website - http://edu.of.ru/volsch31/default.asp?ob_no = 2300.

The table provides a list of characteristics of some tools for creating testing systems and compares them.

In relation to this type of software, it is very difficult to consider efficiency, because The influence of the person (the teacher creating the tests and the student answering the test) is great. If efficiency is considered from the point of view of the speed of testing tests, then this indicator largely depends on the speed of information transfer via computer network, on the number of test tasks.

As can be seen from the above description and comparison of capabilities, the considered software products can be used for testing in various disciplines, both humanities and technical (due to the ability to work with text, graphics, music, multimedia).

If you give 1 point for the presence of each feature, then it turns out that of the systems under consideration, MOODLE received 22 points, UniTest System - 15, Test Designer - 11, INDIGO - 14, VeralTest - 12 (Express version) and 16 (Professional version).

Taking into account the presence of a system for setting up the order of submitting tests, a system for interaction via a computer network and other factors, the MOODLE, INDIGO and VeralTest systems have the most advanced capabilities. It is these systems that are most often used in practice when testing students.

Conclusion

Evaluation of software quality indicators can be carried out using various methods and methods. The quality assessment methodology presented in the article, based on the principles of the ISO 9126 standard, allows you to:

Assess the quality of software systems using various systems of quality indicators;

Ensure the quality of programs when developing technical specifications and control it at all stages life cycle, i.e. assess the minimum level of quality with incomplete information about software systems, which is achieved with the already obtained values ​​of quality indicators;

Comparative characteristics of some tools for creating training courses

	Name			"Test Designer"		VeralTest Express/Professional
	Reliability
	Completeness (probability of failure)					Low / High
	Failure tolerance (operability)
	Recoverability
	Availability of the system Reserve copy
	Saving tests to separate file
	Ease of use
	Ease of learning
	Availability of guidelines for studying
	Understandability
	Availability ready-made templates tests		+ (in English)
	Availability of expanded help system
	Convenience and ease of use
	Availability of a menu (button) for creating a test
	Working with graphics
	Working with sound
	Creating Control Buttons
	Possibility of automatic answer evaluation
	Setting deadlines for answering questions
	Availability of a function for determining the time to answer questions
	Time limit for answering each question
	Limiting the total test time
	Ability to divide questions by difficulty level
	Functionality
	Availability of security measures (for example, test encryption)
	Ability to operate a local computer network
	Work in Internet networks
	Ease of maintenance
	Availability of service technical support
	Availability of separate modules
	Availability of settings for the engineer
	Availability of settings for the teacher
	Availability of settings for the person being tested
	Portability
	Availability network version				+ (internet)
	Occupied volume					20.8 MB / 60 MB

Based on installed system quality indicators, evaluate different programs for the same purpose in order to identify the best of them.

When developing indicators that evaluate test creation systems, the features of this type of program were taken as a basis. In the future, you can expand the list of indicators under consideration and change them according to the characteristics of other types of software.

Reviewers:

Dorrer G.A., Doctor of Technical Sciences, Professor, Head. Department, Federal State Budgetary Educational Institution of Higher Professional Education "Siberian State Technological University", Krasnoyarsk;

Levshina V.V., Doctor of Technical Sciences, Professor, Head. Department of Quality Management and Mathematical Methods of Economics, Siberian State Technological University, Krasnoyarsk.

The work was received by the editor on 05/07/2013.

Bibliographic link

Gorbachenko I.M. ASSESSMENT OF SOFTWARE QUALITY FOR CREATION OF TESTING SYSTEMS // Fundamental Research. – 2013. – No. 6-4. – P. 823-827;
URL: http://fundamental-research.ru/ru/article/view?id=31642 (access date: 07.20.2019). We bring to your attention magazines published by the publishing house "Academy of Natural Sciences"