Nullstone Logo


Request Information

Release Notes
PGP Information
Service Report
Write Us

Performance Results
Glossary of Terms

Compiler Connection
Compiler Jobs

NULLSTONE for C Release Notes

Release 5.6

  • Volatile Tests: The original volatile tests were developed using the Posix alarm() function. Microsoft Visual Studio and the Windows environment do not support alarm(). The volatile tests have been ported to the Windows environment using _beginthread() and Sleep() as an alternative to alarm(). The volatile tests have also been expanded to include additional tests.

Release 5.5

  • Intel ICL Compiler: Add configuration files for the Intel ICL compiler.

Release 5.4

  • driver -x path: Implement the 'driver -x path' option to specify the location of the NULLSTONE suite so that multiple instances of NULLSTONE can be run from other directories. See the example configuration in the directory config/npath.

Release 5.3

  • Code/Data Size: Add support to measure both code size and data size. "commands" file has been enhanced to include both codesize and datasize strings. Results database has been enhanced to store both size measurements. Analsysis report generator has been enhanced to generate both code size and data size reports.

Release 5.0.1

  • User Guide: Include the revised User Guide.

Release 5.0

  • C99 data types: Add support for C99 data types _Bool, float _Complex, double _Complex, long double _Complex.
  • C99 restrict qualifier: The C99 restrict qualifier can be used to provide hints to the optimizer. A new test category has been added to measure the optimization of the restrict qualifier.
  • DSP-C: Expand the tests for DSP-C.
  • Alias Optimization (scalar vs array): Indexed pointers can not point to scalar objects, and this information can be used to ehnahce alias optimization of pointers to arrays vs pointers to scalars. An new test category has been added to measure this alias optimization.
  • Improve Compile Time Accuracy: Given that some hosts are now 2.0GHz and faster, the compile time of each test is often less than one second. In previous releases, the compile time was reported as an integer in seconds, which was truncated to zero when less than one second. This release uses more accurate timing information when available on the host, and the compile time is reported as a floating point number.
  • Enhance commands file: Add new fields in the commands file to specify the architecture, model, languages, and features to better specify which tests are to be compiled and executed for each target compiler. Moving this information from the config.h files means that driver is no longer hard wired for each target compiler, and does not have to be recompiled to test each new target compiler.

Release 4.8

  • Implement C99 long long int format specifier: Use the C99 long long int format specifier "%lld" when printing long long int values.
  • Fix defect in if_xx tests: Fix a defect in the conditional expression in the tests if_12.c, if_17.c, if_22.c, and if_32.c.
  • Fix defect in unroll_2.c: Fix a defect in unroll_2.c to use the correct loop count in the ideal encoding.
  • Add Sandbridge Configuration: Add configuration files for the Sandbridge compiler.
  • Add GreenHills Configuration: Add configuration files for the GreenHills compiler.

Release 4.7e

  • Quoted strings: Add support for quoted strings within the compile and execute strings in the commands file.
  • Path Translation: Add the -p option to the NULLSTONE driver to override DOS, UNIX, and Mac path translation.
  • Bash Shell Script Generation: Add the -b option to the NULLSTONE driver to generate a Bash script that will compile and execute the tests. Some compilers and/or tool chains can not be invoked from another process and therefore the NULLSTONE driver can not directly call these applications using fork() or similar mechanism. The generated Bash script can be used to run the suite (i.e. invoking the compiler and executing the tests) instead of the NULLSTONE driver doing same.
  • Return value in loop tests: The function f() in loop_12, loop_13, loop_14, and loop_15 has no return value. Change the function type to void f().
  • HiTech Z180 Configuration: Add configuration files for HiTech Z180 compiler.
  • Cygwin Configuration: Add configuration files for cygwin compiler and environment.
  • Fclose() Defect: In the file driver.c, check to make sure stream is open before calling fclose().

Release 4.7d

  • Main() Initialization Code: Add a define MAIN_INIT to support embedded compilers that require a call to initialization code upon entry to main().
  • Cradle Technologies Configuration: Add configuration files for Cradle Technologies compiler.

Release 4.7b

  • DSP C Data Types: Add DSP C fixed point data types __fixed and __accum and short, long, and unsigned variants of these types.

Release 4.7a

  • Analysis Command Line Options: Implement -source and -data command line options so that source files and data files can be input to the Analysis tool on the command line.
  • Loop Initialization: Initialize a loop counter in code size test size_44 so that the test will execute.
  • Integer Overflow: Change input constants in combine tests comb_03, comb_07, comb_33, comb_37, comb_47, comb_77, and comb_78 to avoid interger overflow.
  • New Target Configurations: Add configuration files for Diab, GCC, Infineon, and KAI compilers.

Release 4.7

  • NO_FLOATING_POINT: The new define NO_FLOATING_POINT is supported in the config.h file for target architectures that do not support floating point.
  • Analysis source file: The analysis command has been enhanced to accept a source file on the command line with the syntax "analysis [results.dat] [-s source]".
  • Compile-time Error Messages: If a test fails during compilation, the NULLSTONE driver will collect the error messages generated by the compiler and log them in the results.dat file, and the analysis tool will display the error messages in the "show detailed failure" report.
  • Fix defects: Defects in the tests size_30.c and if_07.c have been corrected. Typographical errors in the on-line documentation have also been corrected.
  • Add new config files: Add configuration files for the HP and Tasking TriCore compilers.
  • Double to unsigned long conversion: In the timing routine, there is a conversion from floating-point to an unsigned long. At least one compiler had problems with the implicit type conversion, so an explicit type cast has been added to avoid the defect.
  • Expanding empty ${var} in driver: If the first component in a command was ${var} and getenv() returned an empty string, the function expandCommand() in the driver would incorrectly compute the wrong string length. The buffer is now initialized to a null string at the beginning of the expandCommand() function to prevent this problem.

Release 4.6

  • Analysis Command Line Argument: The Analysis tool can open a file given on the command line.
  • Support ${ENVIRONMENT_VARIABLE} in command strings: The command strings in the command file now supports ${X} style environment variables.
  • Enhance timing error program to support 64-bit timing algorithm: The timing error program (error.c) has been enhanced to support the 64-bit timing algorithm. This program is useful to measure timing error (i.e. variations in timing from run to run).
  • Makefile Configuration Files: The sample makefiles for target architectures and compilers are now generated using a preprocessor for easier porting to new targets.
  • Parallel Test Execution: The NULLSTONE driver has been enhanced to support parallel test execution. This feature significantly increases the turn-around time to run the suite in environments that have multiple CPUs and/or platforms.
  • Main() function arguments: Some target compilers can not support main() function arguments. An #ifdef has been added to support these targets.
  • Fix default type: The default type for the test src/combine/comb_19.c was incorrect and has been fixed. This did not affect normal execution of the test when compiled by the NULLSTONE driver, but could result in a failure when compiled on the command line without a -Dtype argument.

Release 4.5

  • Data Width Specifications: The suite has been enhanced to provide better support for 8, 16, 32, and 64-bit processors and compilers. Each test now includes a specification for the data width requirement for each data type used in the test. The syntax of the commands file has been enhanced to include the data width definitions for each compiler. The NULLSTONE driver will compare the data width definitions for each compiler with the data width specification for each test, and only select those tests that are appropriate for that compiler.
  • Enhanced Native Timing Algorithm: The clock speeds of today's processors are reaching 400MHz and more. When counting the number of iterations that can be executed within the timing loop, it is possible on some processors to exceed a 32-bit integer. The timing algorithm has been enhanced with a 64-bit integer using two 32-bit long integers.
  • Enhanced Embedded Timing Algorithm: When testing a compiler using a simulator or an embedded platform with precise timing registers, it is not necessary to execute the test kernel for multiple iterations in a timing loop. Instead, the kernel is only executed once. In this case, the overhead of the clock() function may be much larger than the time actually spent in the kernel(). The timing algorithm has been enhanced to subtract out the overhead of the clock() function from the measured time, thus providing more accurate timing analysis.
  • File Name Globbing: The NULLSTONE Analysis tool has been enhanced to support file name globbing in the open command. For example, it is possible to open multiple results files using the command:

    open *.dat


Release 4.3

  • On-line Documentation: The NULLSTONE User Guide has been converted to HTML format, and can be accessed by any HTML browser. The User Guide can be accessed by the file html/usrguide.htm.
  • Read Compressed Files: The Analysis Tool has been enhanced to read compressed files using compress, gzip, or pkzip. The compression ratios for NULLSTONE performance database files is quite high, and compressing these files can save a considerable amount of space.

    To open a compressed file, use the 'open' command on a file that ends with .Z, .gz, or .zip.

  • 64-bit long long support: The support for 64-bit long long integers has been enhanced to support other keyword definitions, such as __int64 and other user-specified spellings.
  • Enhanced Timing Loop: Some compilers support link-time optimization, and in previous releases, these compilers could delete the code from the timing loop for some tests. The timing loop has been enhanced with a store to a volatile variable so that the code in the timing loop can not be removed.

Release 3.9b3

  • C++ Compatibility: The NULLSTONE tests and supporting library routines are now compliant with C++. In theory, the performance of the tests should be that same when compiled with C and C++ compilers that use the same optimizer, but often are not in practice. Customers are encouraged to test using both C and C++ compilers.
  • Analysis Stack Size: The Analysis Report Generator contained a call-by-value of a large structure that should have been call-by-address. This could cause a core dump on systems with a limited stack size (i.e. DOS environments).
  • Missing Argument: A function call in Analysis was missing an argument and caused a core dump when generating Code Size reports.
  • HTML Table Headers: One of the HTML table headers was misspelled, which caused it not to be linked to a URL on the Nullstone web pages.
  • Table Width: The default table width was 80 columns, which caused double-spacing on displays that only supported 80 columns. A new command "set pagewidth" has been implemented to support narrow displays.
  • Integer Overflow: When calculating percent differences in the Analysis tool, very large differences in Nullstone Rates could cause an integer overflow, and report an incorrect percent difference.

Release 3.8.8

  • HTML Table Generation: The NULLSTONE Analysis tool can generate reports using HTML tables, making it possible to review the performance data and other reports using a web browser.
  • Data Export Capability: The NULLSTONE Analysis tool can export the performance and other data in a format suitable for import into spreadsheet programs and other applications.
  • Conformance Tests: In previous releases, the NULLSTONE Conformance tests, which evaluate correct compilation and execution, were included in the Performance reports (as either 0% or 100%). In this release, the Conformance tests are not included in the Performance reports, and are only included in the Failure reports.
  • NULLSTONE Difference Report: The NULLSTONE Analysis tool can now generate NULLSTONE Difference reports based on either the NULLSTONE Rate or NULLSTONE Ratio. Previous releases only supported the NULLSTONE Ratio.
  • Self Tests: A collection of NULLSTONE Self Tests have been added to the suite to make sure the NULLSTONE tools and suite have been correctly installed and built.
  • Alias Analysis: A defect in the performance test alias_12.c has been corrected.
  • Unsigned Short A defect in the header file which affected tests with data type unsigned short has been corrected.
  • Bimodal Performance: The Bimodal Performance of some of the tests have been refined, which reduces the possibility of false positives.
  • Code Size Analysis: A collection of tests have been added to the NULLSTONE suite to measure code size. This feature is particularly important for evaluating embedded compilers. The NULLSTONE user provides a program that extracts the code size from the target executable, and the NULLSTONE Driver places the information into the performance data for analysis in the NULLSTONE Analysis tool.

© 1990-2012 Nullstone Corporation. All Rights Reserved.