Last updated: $Date: 2008-05-21 10:04:46 -0400 (Wed, 21 May 2008) $ by $Author: john $
Contents
7. The submit command may fail under Windows Vista
8. Your vendor has not defined POSIX macro WEXITSTATUS with invalid flags file
9. Error: corrupt result file; unable to format after failed FDO build
10. WARNING: accessor 'benchmark' not found during reportable run
11. Runaway specmake.exe on Windows
12. Style settings may appear ineffective when formatting results with multiple flags files
14. WARNING: accessor 'setup_error' not found with minimize_rundirs
15. Index utility prints Expected 17 fields; got 16 (base_copies, ...
1. Missing example-advanced.cfg
2. Parallel builds aren't parallel on Windows
3. Reporting of seconds
4. Unexpected message installing on Solaris/x86
5. Incorrect path example in shrc.bat on Microsoft Windows
6. The mailcompress option doesn't work yet
13. ERROR(update_config_file) with src.alt.
This document describes known problems in SPEC CPU2006. The latest version of this document may be found at http://www.spec.org/cpu2006/Docs/errata.html.
If you are looking for the solution to a technical question, you might also want to review the latest version of http://www.spec.org/cpu2006/Docs/faq.html.
SPEC CPU2006 is extensively tested on a wide variety of systems prior to release, including an assortment of Unix and Linux systems, and several versions of Windows systems; big-endian and little-endian systems; 32-bit and 64-bit systems. Nevertheless, as with any software product, problems may be found too late in the testing process to be fixed in the release, or may be discovered after release.
If you discover problems that are not listed here or in the FAQ, please report them to SPEC.
About the format of this document:
Notice to maintainers: When updating this document, please do not ever re-use problem numbers.
Problem summary: A common submit command that works normally under Windows XP may fail under Windows Vista.
Details: The following way of binding processes to processors is often used on Windows XP:
submit=specperl -e "system sprintf qq{start /b /wait /affinity %x %s}, (1<<$SPECCOPYNUM), q{$command}"
Under Windows Vista, the above command sometimes results in the benchmark program not executing (no output files are generated in the corresponding run directory). Apparently, cmd.exe is not started.
Workarounds: There are several possible ways to work around this issue:
1. Explicitly invoke cmd.exe:
submit=specperl -e "system sprintf qq{cmd.exe /E:on /D /C start /b /wait /affinity %x %s}, (1<<$SPECCOPYNUM), q{$command}"
2. Or insert an @ and a space before start, which is the Windows syntax for not echoing a line. For reasons that are not completely clear, this does result in cmd.exe getting called.
submit=specperl -e "system sprintf qq{@ start /b /wait /affinity %x %s}, (1<<$SPECCOPYNUM), q{$command}"
3. Or, split the submit command into several lines as follows:
submit0 = echo @ echo off >runex.cmd
submit1 = specperl -e "system sprintf qq{echo set MYMASK= %x >>runex.cmd}, (1<<$SPECCOPYNUM)"
submit2 = echo start /b /wait /affinity %MYMASK% $command >>runex.cmd
submit3 = call runex.cmd
Problem summary: On Windows, attempting to use rawformat with a flags file that contains invalid XML will cause rawformat to crash. The files being formatted will not be changed.
Details: Before flag files are parsed, they are sent through specrxp.exe for validation. If the validation fails, specrxp.exe will exit with a non-zero exit code. The code in rawformat that handles this case relies on macros from the POSIX module which are not implemented on Windows. The error message is
Your vendor has not defined POSIX macro WEXITSTATUS, used at C:\cpu2006/bin/formatter/flagutils_common.pl line 970
The rawformat process stops, no new files are written, and no existing files are modified.
Workaround: The workaround for this problem is to make sure that your XML is valid before using it with rawformat. When modifying flags files, you can run specrxp.exe manually to ensure that your changes have not caused the XML to be invalid or not well-formed:
C:\spec\cpu2006> specrxp -V -s good.xml C:\spec\cpu2006> specrxp -V -s bad.xml Error: Value of attribute is unquoted in unnamed entity at line 253 char 15 of file:///C:/spec/cpu2006/bad.xml
If the command above exits without printing anything as in the first example above, rawformat will not crash when using that flags file. Otherwise, the error message will point out the line on which the error occured. The XML flags files may have UNIX-style LF-only line endings, so you may need to edit the file with wordpad instead of notepad. Note that the path given to specrxp must NOT contain the drive letter specifier; specrxp interprets any filename that begins with letters followed by a colon as a URL.
Problem summary: Certain errors building a benchmark with feedback may cause the run to terminate with Error: corrupt result file; unable to format.
Details: In order for this error to occur, all of these conditions must be true: (1) A benchmark build fails; (2) The build uses feedback-directed optimization; (3) ignore_errors is set (by the config file option, or from the command line); (4) The run is not reportable.
In this combination of circumstances, when an error occurs during the build of a benchmark that uses FDO, the error information is not properly stored, and this causes runspec to output a raw file that is not syntactically correct. The rawformat utility (which is automatically called by runspec) will refuse to format it, generating Error: corrupt result file; unable to format. No human-readable outputs will be generated from the run.
When ignore_errors is not set (the default), errors in the build process cause runspec to stop, so rawformat is not run, and the error message does not occur. If ignore_errors is set, either by using ignore_errors = 1 in the config file or by using --ignore_errors or -I on the command line, runspec will carry on and run all benchmarks that did not have build errors, eventually calling rawformat, which protests the incorrect information in the raw file.
Since ignore_errors is automatically unset for reportable runs, this error will never affect a result that would have otherwise been publishable.
runspec --action build <other options> <benchmarks> runspec --nobuild <other options> <benchmarks>
Problem summary: Under certain conditions during a reportable rate run, the following warning message will be printed. It can safely be ignored.
WARNING: accessor 'benchmark' not found; called from Spec::Config::copies
(call at config.pl line 1372) for object Spec::ConfigDetails: This problem occurs only when all of these conditions are true: (1) The run is reportable. (2) It is a rate run. (3) The number of copies is specified in the config file and not on the command line. (4) A value for parallel_test is not provided in the config file nor on the command line.
In these circumstances, by default the test and train portion of the run are done in parallel, with the number of jobs equal to the number of copies. The runspec process looks up the number of parallel jobs that can be run, and does not find the correct value, and prints the above message.
The message can safely be ignored; in the worst case the test and train runs will be done serially.
Workaround: You can safely ignore the warning, or you can pick from one of these workarounds:
parallel_test = $[copies]
Problem summary: If runspec is interrupted at the wrong time, the currently running copy of specmake.exe will start to consume 100% of one processor and will never exit.
Details: When checking pre-built binaries to see if they are up-to-date, runspec uses specmake.exe to generate options lists for each benchmark. If runspec is interrupted during this process (by pressing Control-C in the command window), sometimes the specmake.exe process that is generating the options lists will enter an infinite loop and consume all of a processor until it is manually terminated.
Normally when runspec is interrupted by Control-C, it prints a message saying that it has been interrupted by SIGINT, and then Terminate batch job (Y/N)?. When this problem occurs, only the SIGINT message is printed, as in this output:
[...] Compiling Binaries Up to date 998.specrand base cpu2006.ic10.1.win32.core2.rate.exe default Up to date 999.specrand base cpu2006.ic10.1.win32.core2.rate.exe default Terminating on signal SIGINT(2) C:\spec\cpu2006>
Workaround: To avoid the problem, do not interrupt runspec during the "Compiling Binaries" phase of the run.
If you do interrupt, and if a specmake.exe process goes out of control, terminate it using the Windows Task Manager. This can be done by opening Task Manager, selecting the "Processes" tab, and sorting the results by "CPU" in descending order. The runaway specmake.exe process should sit at the top of this list; highlight it and press the "End Process" button.
Problem summary: In a per-result flags report, the <style> section may seem to have no effect.
Details: When results are formatted with multiple user flags files, and two or more of those files contain style sections, the resulting HTML output in the per-result flags report will have a <style> section that many browsers will ignore. When the CSS sections of multiple flags files are merged, the SPEC tools separate them with comments that note the provenance of the settings in question. The problem is that the notes are in HTML-style comments which are not valid for CSS. This causes most browsers to simply ignore the CSS in the affected document.
Note that for results that will be published at the SPEC website, multiple <style> sections will work correctly (the copy of the formatter there has been fixed).
Workaround: There are two possible workarounds:
Make sure only one of the flags files has a style section. This is the simplest course of action.
If you really want multiple <style> sections (e.g. you will be publishing your results at the SPEC website where the problem has been fixed), you can fix your local HTML-style comment markers by hand. Edit them to replace HTML-style comments with CSS-style markers. For example, the generated CSS section will have comments such as:
<!-- CSS section from <filename> -->Those may be deleted, or replaced with something like
/* CSS section from <filename> */
Problem summary: During the benchmark run the tools may print messages such as:
WARNING: accessor 'setup_error' not found; called from Spec::Benchmark::bzip2401::setup_error (call at runspec line 939) for object Spec::Benchmark::bzip2401 WARNING: accessor 'setup_error' not found; called from Spec::Benchmark::bzip2401::setup_error (call at runspec line 953) for object Spec::Benchmark::bzip2401
Details: This prooblem occurs when the minimize_rundirs feature is used, and the run is not reportable.
The normal setup process adds a flag to each benchmark object indicating whether there was an error during setup. When 'minimize_rundirs' is used, a portion of normal setup is skipped and the flag is not added. Note in the text above that both lines 939 and 953 may be referenced in runspec. At those lines, the complaint is not about whether the flag is true or false, just that it doesn't exist. The messages are harmless and do not indicate any error. They can safely be ignored.
Workaround: Ignore the error, or do not use the 'minimize_rundirs' feature.
Problem summary: The index utility fails, complaining about a missing field
Details: The index utility is officially unsupported. Nevertheless, some people have found it useful when building directories of results (e.g. for internal use, or prior to submitting results to SPEC). With V1.1, you may see messages such as:
$ $SPEC/bin/scripts.misc/index Expected 17 fields; got 16 (base_copies, basemean, hw_model, hw_nchips, hw_ncores, hw_ncoresperchip, hw_nthreadspercore, hw_vendor, nc, nc_is_cd, nc_is_na, peakmean, rate, test_sponsor, tester, units). Skipping result in ./fprate2.ref $
The reason that the problem occurs is because of the new features added relatively late in the development of V1.1 to automatically set the field "Parallel: Yes/No". The index utility that ships with V1.1 has not caught up with that change.
Workaround: The following edit removes the index utility's expectation that the parallel field will be found in the .rsf file.
$ diff -u $SPEC/bin/scripts.misc/index $SPEC/bin/scripts.misc/index2
--- /bench/cpu2006/v1.1/bin/scripts.misc/index Tue Feb 12 18:44:01 2008
+++ /bench/cpu2006/v1.1/bin/scripts.misc/index2 Fri Jun 6 12:12:40 2008
@@ -130,7 +130,6 @@
$vendor_field,
'test_sponsor',
'tester',
- 'sw_auto_parallel',
'hw_model',
'hw_nchips',
'hw_ncores',
@@ -157,7 +156,6 @@
[ 'Test Sponsor', 'test_sponsor', 1, 2, ],
[ 'System Name', 'hw_model', 1, 2, ],
[ 'Base<br />Copies', 'base_copies', 1, 2, ],
- [ 'Auto<br />Parallel','sw_auto_parallel', 1, 2, ],
[ 'Cores', 'hw_ncores', 4, 1, 'Processor', ],
[ 'Chips', 'hw_nchips', 1, 1, '', ],
[ 'Cores/<br/>Chip', 'hw_ncoresperchip', 1, 1, '', ],
$
Details: For certain values, the SPEC tools print 3 significant digits. This is intentional. For example, if one system has a SPECint_rate2006 performance of 1234.456 and another has a SPECint_rate2006 performance of 1229.987, it is arguable that the performance of these systems is not materially different. Given the reality of run-to-run variation (which is, sometimes, on the order of 1%), it makes sense to report both systems' SPECint_rate2006 as 1230.
There was agreement that it is acceptable to round SPEC's computed metrics to 3 significant digits. However, it has been noted that the argument is weaker for rounding of original observations. In particular, if we wish to acknowledge the reality of run to run variation, then it seems reasonable to report a time of 1234.456 seconds using an integral number of seconds (1234), rather than rounding to the nearest 10 seconds (1230).
Results posted on SPEC's web site in reports such as the HTML, PDF, and text formats will use 3 significant digits for computed metrics, but seconds larger than 1000 will be reported as an integral number of seconds. A future maintenance update of SPEC CPU2006 will behave the same way. But for SPEC CPU2006 V1.0, you may notice that reports you generate for yourself round seconds to three significant digits.
Resolution: Fixed in CPU2006 V1.1
./install.sh
SPEC CPU2006 Installation
Top of the CPU2006 tree is '/data1/kumaran/kit97'
specbzip2: Cannot find /lib64/ld-linux-x86-64.so.2
Killed
specbzip2: Cannot find /lib/ld-linux.so.2
Killed
There appears to be only one valid toolset:
solaris-x86 Recommended for Solaris 10 and later
Built on Solaris 10 with Sun Studio 11
ERROR(update_config_file): Config file line at 36 (prepared_by) does not match tester! ERROR: Edit of stored config file failed; Update of tester failed.
Copyright (C) 2006, 2008 Standard Performance Evaluation Corporation
All Rights Reserved