nyenaceyg
Dołączył: 13 Gru 2010
Posty: 522
Przeczytał: 0 tematów
Ostrzeżeń: 0/5
Skąd: England
|
 Wysłany: Czw 18:53, 14 Kwi 2011 Temat postu: _152 coverage testing embedded software |
 |
|
Coverage testing embedded software
Coverage testing embedded software
Abstract: The coverage test is to verify the correctness of software functions and structure of the problem is very important to find ways and means, it is a tool to Jiezhu some good results can be achieved, software quality and time to meet the dual demands (pure manual testing workload, is not convenient, long cycle). How to make good use of this more mature tools and adaptability of its mechanism of transformation is very important. This article focuses on the working mechanism of such tools, as well as the special requirements of embedded software testing, and embedded operating systems on proprietary tests as an example.
Keywords: embedded operating system coverage testing software testing tools
1 Overview
Software testing is a very broad concept. Throughout the software life cycle from the entire process of point of view, the test can be divided into the module testing, integration testing, system testing phase. Static checking and testing can be divided into two major categories of dynamic running tests. In the dynamic operation test, but also a white-box testing based on program structure (or coverage testing), and based on functional black box testing. Not only concerned about the function of the test procedures, as well as of a test, strength testing and so on.
To achieve better test results, we must have comprehensive test plans, control testing process, testers experience, but also need the help of some effective aid, especially in today's increasingly large scale software, testing have multiplied the case. Corresponding to the classification of the above tests, the test tools can be divided into: the source code to support static rule checking and quality assessment of static analysis tools,[link widoczny dla zalogowanych], support for dynamic overlay program unit testing tool, the overall operation of the software system to test the performance tool. In addition, there are some special-purpose or special tools, such as the protocol tester, the memory test tools. These tools are more mature commercial products, but also developed a way to get through.
This specific discussion of a special kind of system software - embedded real time operating system - to cover the testing situation. Concerning the characteristics of this type of software, testing difficulties and characteristics of the adaptive transformation of existing test tools and test examples.
2 software coverage testing
Cover is a white box testing methods, testing procedures must have a list of specifications and procedures to the internal structure of the program is based, to design test cases. The basic criterion is the test case to cover as much as possible within the logical structure of the program, found errors and problems. Therefore, the coverage test typically used in software testing early, that unit testing phase.
Cover several methods or strategies as listed in Table 1. Table 1 Some typical coverage strategy
Coverage strategy
Definitions
Statement coverage
In the development of test cases, so that each statement in the program are executed at least 1 times. The disadvantage is that some logic errors can not be found
Decision coverage
Implementation of adequate test cases, each judge makes the program are given a Condition coverage
Implementation of adequate test cases, making the determination of each condition to obtain a variety of possible outcomes
Decision / condition coverage
Implementation of adequate test cases, making the decision made in every condition possible values, and makes possible for each to determine the results achieved
Condition combination coverage
Implementation of adequate test cases, so that each decision in terms of various combinations of at least 1 or more times. Its characteristics are covered more fully meet the conditions for combined coverage of test cases must also satisfy decision coverage, condition coverage and decision / condition coverage.
Can be seen from the above brief introduction, these types of procedures covered by the strict policy following trends: other coverage strategies include: modified condition / determine the coverage (usually referred to as MCDC), path coverage, function coverage, call coverage, line of the code sequence and jump coverage, data flow coverage, object code branch coverage, loop coverage, relational operator coverage. With the growth of software size to achieve a comprehensive coverage of the number of test cases required more and more large, so the characteristics of software objects under test to select the appropriate coverage strategy is very important; the same time, the test to determine a reasonable target 100% coverage often pay a high price, and other methods should be combined with the formal review to find more software failures.
3 coverage testing tool
To obtain better coverage test results, it needs certain tools. These tools generally have the following features may make up for human testing of the defects:
① the internal structure of software to help formulate strategies and design test cases covering;
② compiler with the appropriate combination of software implementation of automated test instrumentation, in order to cover its operation to generate information and the information collected;
③ According to the calculation of coverage coverage information collected to help the tester find the parts not covered by the software to improve the test cases to improve coverage.
Dynamic in the use of coverage testing tool, you need three elements: the test case, test code plug had to gather information and analysis covering the tool itself. Code instrumentation done automatically by the tool, through the implementation of test cases, and then covered by the tools to collect and analyze information, you can see the coverage of indicators. Figure 1 shows the basic process of achieving coverage test. 4 principles of embedded software test coverage
Embedded software development and general software big difference is, we need a way to cross development: development tools run on the host hardware and software configurations on the rich, while the embedded application software and hardware resources running on the relative lack of target machine . Testing of such software for the same problem also exists: the test tool is running on a host to test the information needed to generate the target machine, and through some physical / logical connection through to shrink host, received by the test tool. Therefore, the embedded software testing is an important issue is building between the host and the target's physical / logical connection, data transmission problems to solve.
Embedded software is covered by the basic principle of the test shown in Figure 2.
The target side, plug off the application under test will cover the information sent to the message queue, a special task at the appropriate time will be responsible for the information sent to the host side. Reduce the host side has a special module to receive coverage information. And analysis tools to analyze and online coverage of the growth dynamic display.
Coverage testing embedded software support tool should address the following two key issues:
* The combination with the embedded operating system coverage testing tools and embedded operating system combination is reflected in three aspects. First, the target side, the application tasks and specifically responsible for the collection / information covered by the task of uploading the message queue to pass through the data, the message queue can use the embedded operating system, the appropriate mechanism to achieve. Second, the special task can also be viewed as a special application tasks must also be embedded operating system support, because the task of management is one of the basic functions of the latter. Finally, the target machine and the host can be used between the serial port or Ethernet communication mode, the driver of the serial or network protocol can be used embedded operating system, the corresponding component.
* Embedded cross-development tools and other relations
The development of embedded applications usually means cross development, almost all of the development tools are 3 parts to solve the problem: the host part of the function, the target part of the function of host connectivity issues with the target. Which host the connection with the target is a bottleneck, if the different tools to use the same physical line for data transmission, will have to address the physical line (or the hardware port) the correct share. For example, in Figure 3 shows the environment, the host side tools through a unified interface - the target server (target server) to achieve access to the communication line, the target side of the debug agent (debug agent) is a variety of information (debugging information, coverage information, time information, object information, etc.) the core of the collection and delivery.
5 Logiscope in the embedded operating system DeltaCORE Testing
Logiscope is the company's CASE products Verilog, software coding, testing, maintenance of a wide range of services and support coverage testing embedded software.
5.1 Preparation before test
Preparation before the test is the support of the testing done on DeltaCORE transplantation.
Currently, Logiscope has some mature commercial embedded operating system support, such as pSOS. DeltaCORE is our strong independent development of embedded real-time operating system kernel, in order to achieve using Logiscope application DeltaCORE even DeltaCORE own testing, we mainly solved as described in Section 4 1 a key issue. In order to support embedded applications testing, Logiscope offers side running on the target machine code (or target side of the support library), which includes:
* One to collect and send information covering the main loop thread, the thread that is the special tasks in embedded applications;
* Function to achieve specific data, including serial or network drive, they will be called the thread;
* Plug the realization of functions, these functions are tested code calls, to cover the message block into the buffer;
* Information on the buffer queue management;
* Initialization code.
For example, when a test run into if (... ...) statement, the process shown in Figure 4.
To support DeltaCORE of tests associated with these mechanisms code for transplant, including the following:
* Information will be collected and send cover the main loop thread as the target-side application running on a special mission;
* The serial port driver by LambdaTOOL BSP (board support package) instead of the serial port driver, the driver of the network, using the matching component DeltaNET DeltaCORE driver in the implementation;
* Use DeltaCORE mailbox message queue mechanism to realize the creation and management, plug-mail sent to those covered by the code message block;
* The root of the application task DetaCORE call Logiscope the initialization function, to create a special task-mail purposes.
DeltaCORE application development, we used their complete development tools LambdaTOOL. Because the tools used to achieve the target server version does not (target server) in the debug mode, so the use of physical port used exclusively, the debugging tools and other tools can not share the same port. We can try to use the network to upload and start target application, and coverage of information transmitted via the serial port.
5.2 DeltaCORE coverage of the testing process and results
For the function of internal, Logiscope support coverage strategy are:
* Command block IBs (Instruction Blocks)
* Judge to determine the path DDPs (Decision-to-Decision Paths)
* MCDC (Modified Condition / Decision)
Support at the project level coverage policy is:
* Process the path to the process of PPP (Procedure-to-Procedure Path)
In DeltaCORE the test, we used a more common range strategy - to determine the path to the judge, its meaning is: DDP is an instruction sequence, which is a function or to determine the starting point (if, while, ... ...) of the entry point , its exports to determine the next function or exit point, between the judge can not have, such as in Figure 5 includes five DDPs:
The specific process of testing:
① Cartridge Analyzer using the source code of DeltaCORE instrumentation, and generate the instrumentation information file.
② after transplantation Logiscope the target process and instrumentation after the end of the kernel source code compiled and linked together into a library, to replace the original core library, available to the application.
③ write test cases, from the application point of view to achieve the various system functions using DeltaCORE call, seeks to determine the traversal of all branches of the kernel function, and these cases compiled into an executable program.
④ start in the host-side coverage information collection and analysis procedures used LambdaTOOL debugger to download and launch applications. DeltaCORE coverage information is passed to the host, the dynamic analysis program shows the growth in coverage, and the information recorded in a file.
⑤ After application execution, start Logiscope ex post analysis tools, will cover the information log file information file with the plug (in the source code instrumentation subsidiary in the resulting file) were compared, to help test the staff clearly understand the function of each test path within the coverage, to improve the test case for help.
⑥ test to modify test cases and re-testing process throughout; the test results can be superimposed, coverage will be increased.
After 2 months of time, we DeltaCORE 1.1 version of a total of 79 files covering 115 function tests, coverage has reached 70.55%. Preparation of 89 test cases, the main function of the 60 have a higher API coverage, coverage of 100%, about 51.3%.
6 Summary
We use Logiscope tools for embedded real-time operating system DeltaCORE coverage test carried out to achieve a good coverage; discovered and dealt with some defects and improve software quality and reliability, but there are also disadvantages:
① test should be a good plan, including the choice of the test sequence, test case design, test, document management.
② Since the test depends on the operating system means the mechanism, and the measured object is the operating system itself, so some of these mechanisms have not been instrumentation and test the code, otherwise an error occurs. For example, the operating system's initialization function os_init, before this function is finished running, the operating system of the corresponding mechanism has not yet established, so it will cause problems for instrumentation, can not properly get coverage information. As another example, for efficiency considerations, and the relevant part of the system clock function is not instrumented, because the program is running, the clock is the most frequently produced by an external event, if the plug, it will generate a lot of coverage information, the information would be cached, transmission, collection and processing pressure. In addition, the use of the tool does not support the function of the plug assembly and testing. Taking these various reasons, DeltaCORE 1.1 overall coverage still appears relatively low, you need to use other methods to improve it. For non-operating system components and application of the test, the absence of the operating system itself, so is expected to reach a higher coverage.
③ The method can not be used time performance testing. It is pure software testing methods, a large number of data generation, transmission and collection procedures on the measured interference, can only do white-box verification of the functional structure. If you do performance testing, methods should be used with certain software and hardware tools, such as CodeTEST.
Testing of embedded software products are many, in addition to coverage tests, there is still time performance test, memory test and analysis used is an important subject of our study.
Post został pochwalony 0 razy
|
|
