The Standard Performance Evaluation Corporation (SPEC) is a non-profit corporation formed to establish, maintain and endorse a standardized set of relevant benchmarks that can be applied to the newest generation of high-performance computers. SPEC develops benchmark suites and also reviews and publishes submitted results from our member organizations and other benchmark licensees.
05/27/2016: SPEC plans to bring a performance benchmarking symposium, an energy-efficiency symposium, and several of its technical meetings to Beijing, China, the week of October 24, 2016. This will be an opportunity to bring together computing researchers, policy makers, government agencies, industry practitioners and others interested in performance and energy-efficiency evaluation to learn about SPEC, share ideas, discuss challenges and present results. If you are interested in attending, please complete and submit this pre-registration form.
05/03/2016: SPEC announces the release of SPEC Cloud_IaaS 2016, a new benchmark software that measures the performance of infrastructure-as-a-service (IaaS) cloud implementations. The SPEC Cloud IaaS 2016 benchmark measures scalability, elasticity and provisioning capabilities of IaaS cloud implementations, both public and private. Cloud providers, users of cloud services, and hardware, software and management platform vendors can use the benchmark to evaluate how different implementations affect cloud performance.
03/14/2016: The SPEC Workstation Performance Characterization (SPECwpc) group has added support for Windows 10 to SPECwpc V2.0, its benchmark that measures all key aspects of workstation performance based on diverse professional applications. The benchmark does not require the full application and associated licensing to be installed on the system under test, simplifying set-up, running and results reporting.
02/11/2016: SPEC Symposium'16 - Energy Efficiency will be held on Friday, March 18th, in conjunction with ICPE 2016 at TU Delft, The Netherlands. First presented in the U.S. in January and scheduled to be delivered in Asia later this year, this will be the only opportunity to participate in this event in Europe. A broad range of contributors and presenters from industry and academia have worked diligently to create a detailed overview of the roadmap for energy efficiency as applied to servers. Limited seating is available. If you would like to attend, please register; there is no charge to attend the symposium.
01/27/2016: Yufei Ren from Stony Brook University, New York is the winner of the 2015 SPEC Distinguished Dissertation Award with his dissertation about scalable end-to-end data I/O over enterprise and datacenter networks. The award will be presented at the ICPE 2016 conference.
01/06/2016: SPEC Symposium'16 - Energy Efficiency will be held on Monday, January 11th, in conjunction with SPEC's annual meeting at the Marriott Silicon Valley in Fremont, CA. There will be more than 10 presentations, including SPEC's Energy Metrics and The Future of Energy Efficiency of Servers. If you would like to attend, please register; there is no charge to attend the symposium.
12/01/2015: The deadline for the submission of workshop papers at the 7th ACM/SPEC International Conference on Performance Engineering has been extended. Please visit the ICPE Web site for more information on the workshops and the extended submission deadlines.
11/19/2015: An update to the SPEC SFS2014 benchmark is now available. This update fixes a possible sensitivity to systems that have inline compression enabled and incorporates all errata fixes since the initial release. Going forward, all result submissions must be made using the SP1 version of the benchmark. Existing licensees of V1.0 are entitled to a free upgrade to SP1.
11/18/2015: The SPEC Workstation Performance Characterization Group (SPECwpc) has released SPECwpc V2.0, a major upgrade to its namesake benchmark that improves scalability measurement for workstations with large numbers of processing cores and fully integrates solid-state storage into performance testing.
09/30/2015: SPEC HPG announces an update to the SPEC ACCEL benchmark. The V1.1 update includes bug fixes, improved documentation and PTDaemon 1.7.1 and is available as of Sept. 30, 2015. Results from V1.0 will be accepted until the review cycle starting December 9, 2015. After that date, all submissions must be made using V1.1. Licensees of ACCEL V1.0 should contact SPEC for upgrade information.
09/23/2015: SPEC announces the release of SPECjbb2015, an enhanced Java server benchmark that now supports virtualized hosts, adds stricter performance validation, and specifies faster response times for a more accurate assessment of total capacity in a variety of real-world deployments.
09/23/2015: The SPEC High-Performance Group (SPEC/HPG) has posted a YouTube video previewing a tutorial titled "Towards Comprehensive System Comparison". The session will be held at SC15 in Austin, Texas, on Sunday, November 15, 2015, 1:30 to 5 p.m. in Room 13AB. Through hands-on guidance and live demonstrations, SC15 attendees will discover how to use SPEC/HPG benchmarks for better analysis, evaluation and procurement of next-generation HPC systems.
08/27/2015: SPEC's Application Performance Characterization (SPECapc) project group has released new performance evaluation software for systems running Dassault Systemes' SolidWorks 2015 CAD/CAM application. The benchmark consists of nine graphics and two CPU tests. It is designed to run on Microsoft Windows 7 64-bit platforms.
08/18/2015: SPEC/GWPG previews upcoming benchmarking plans at SIGGRAPH. These include SPECviewperf support for 4K-resolution content, upcoming workstation performance-measurement initiatives, and new application benchmarks.
08/18/2015: SPECpower_ssj2008 run rules change: The upper boundary of Calibration interval 0 has been increased to 105% due to issues with some multi-node configurations. Please check section 2.5.2 of the Run and Reporting Rules for details.
07/08/2015: Call for contributions for ICPE 2016. The main theme for this year's conference is performance in the data center, in particular for hosting cloud applications and services. Authors are invited to submit original, unpublished papers that are not being considered in another forum. See the conference website for submission guidelines and information.
06/25/2015: Call for nominations: SPEC Distinguished Dissertation Award 2015. The SPEC Distinguished Dissertation Award aims to recognize outstanding doctoral dissertations in the field of computer benchmarking, performance evaluation, and experimental system analysis in general. The award is open to dissertations that have been defended between October 1, 2014 and September 30, 2015, and the nomination submission deadline is September 30, 2015.
06/19/2015: Klaus-Dieter Lange, SPEC board member and SPECpower chair, and Karl Huppler, a long-time supporting contributor to SPEC, are the authors of the recently released "Server Efficiency — Metrics for Computer Servers and Storage", the 12th book in the ASHRAE Datacom Series. Huppler calls the book "a valuable guide for choosing energy-efficiency measures for computer servers and associated storage." It can be purchased on the ASHRAE website.
06/11/2015: LIMBO accepted as SPEC RG tool. LIMBO just became the newest member of the SPEC RG's repository of peer-reviewed tools for quantitative system evaluation and analysis. LIMBO is an Eclipse-based tool for handling, instantiating, and extracting load intensity models.
06/03/2015: The SPEC Workstation Performance Characterization Group (SPECwpc) has released an updated benchmark that extends performance measurement from physical to virtualized workstation configurations. The new SPECwpc V1.2 benchmark measures all key aspects of workstation performance based on diverse professional applications. It supports various hypervisors for creating and running virtual machines, and includes GUI improvements.
05/26/2015: Chauffeur WDK V1.1.0 released: the Chauffeur WDK v1.1.0 simplifies workload development for researchers analyzing system performance and power consumption. Targeting both industry and academia the WDK is based on the SERT v1.1.0 infrastructure, providing run-time & platform configuration support to simplify worklet development. The WDK includes PTDaemon integration for power analyzers and temperature sensors, along with data collection, validation and reporting.
03/10/2015: - SERT V1.1.0 released; SERT V1.1.0 is a significant update, adding support for e.g., Oracle Solaris 11 running on SPARC64 and SPARC T4 as well as Intel x86. The user interface has been enhanced to simplify multi-node test configuration, automated range setting support has been added, and it has been further tuned to deliver successful test passes with a wider range of server configurations. Current licensees will be entitled to a free upgrade post-release.
03/10/2015: - An updated release of PTDaemon, V1.7.1, is released with SERT V1.1.0. This release adds support for additional power analyzers, and enables DC power measurement with many of those analyzers.
02/25/2015: - A new version of SPECviewperf, version 12.0.2, has been released. This version adds support for virtualized workstation configurations. Results from version 12.0.2 are comparable to results from 12.0.1, but neither are comparable to previous versions of the benchmark.
02/17/2015: - With the release of SPEC SFS® 2014, SPEC has begun the process of retiring SPEC SFS® 2008. Results for SPEC SFS 2008 will be accepted for review through April 27, 2015 for the review cycle beginning on April 28, 2015 and ending on May 12, 2015. After that time, no further results will be accepted for review or published on the SPEC website, and no further technical support for the benchmark will be provided.
01/07/2015: - SPEC announces the release of SPECapc for PTC® Creo® 3.0. The benchmark provides eight workflows that exercise all aspects of system performance when running the popular product design application. A licensed copy of Creo 3.0 is required to run the benchmark software, which is available for download in both commercial and non-commercial versions.
12/09/2014: - A defect has been identified in the SPECjbb®2013 benchmark suite which impacts the comparability of results. Sales of the benchmark have been suspended and we are no longer accepting new submissions of SPECjbb2013 results for publication on the SPEC website. The Java subcommittee is working to revise the benchmark to correct this defect, add additional validation safeguards, and release a new version as soon as possible. Current licensees will receive a free copy of the new version when it becomes available.
[Older items can be found in the archive.]