It's time to update the FET benchmarks.
Sadly the old benchmark is pretty out of date. You can still see the old results here:
https://lalescu.ro/liviu/fet/forum/index.php?topic=166.msg729#msg729
FET improved pretty much in speed the last years, because:
- the FET algorithm is better now
- FET support multi core CPUs
- Qt 6 is faster than Qt 5
- FET is compiled now with 64 bit instead of 32 bit
- the compilers are better now
The old benchmark also become too easy for new CPUs and it can't be compiled with Qt 6 anymore.
If you are interested in speed improvements you can maybe already view some of my videos:
https://www.youtube.com/watch?v=XxfJAaPRmxo (Speed up by CPU)
https://www.youtube.com/watch?v=MVt0Yb9y8xc (Speed up by Multi Core)
https://www.youtube.com/watch?v=DejTeNM0T6I (Speed up by software)
https://www.youtube.com/watch?v=nEmsf_-f6P4 (Speed up by software)
https://www.youtube.com/watch?v=rl5N_ZyBB6A (Speed up by time limit)
You can download the FET benchmark here:
Benchmark data set file for all users (as zip file. You must unzip it!):
https://www.timetabling.de/download/BENCHMARK/BENCHMARK.fet.zip
Benchmark data set file for all users (unziped):
https://www.timetabling.de/download/BENCHMARK/BENCHMARK.fet
Executable files:
Windows: https://www.timetabling.de/download/BENCHMARK/fet-6.9.6-BENCHMARK.exe
MacOS (ARM): https://www.timetabling.de/download/BENCHMARK/fet-6.9.6-BENCHMARK-macOS-Apple-M1-M2.dmg
MacOS (x86): https://www.timetabling.de/download/BENCHMARK/fet-6.9.6-BENCHMARK-macOS-Intel-x86.dmg
Linux: https://www.timetabling.de/download/BENCHMARK/fet-6.9.6_BENCHMARK-x86_64.AppImage
Source: https://www.timetabling.de/download/BENCHMARK/fet-6.9.6-BENCHMARK.zip
The benchmark rules are not difficult:
1. Download the FET data set file and the needed executable.
2. Run the data set file on a single thread. Wait until you generated at least 3 timetables and report the last "Tables per minute"-value. ((Load the file: FET -> File -> Open) and (Generate: FET -> Timetable -> Generate multiple))
3. Run the data set file on all available threads. Wait until you got at least 3 times the "Tables per minute"-value and report the last one.
4. Run the data set file only with as many threads has you have performance cores. So without SMT, Hyper-threading, and without efficiency cores.
You can see a small manual also in this video:
https://www.youtube.com/watch?v=GQLiak8AkYQ
You can see some detailed benchmarks here:
https://www.youtube.com/watch?v=OMNvgEuyV-g
(This video is more detailed than the following results)
You will find (shorten) results of the single core and the multi core benchmark results in the next post. Please write your benchmark results here.
The precompiled versions use Qt 6.5.3
Windows uses MinGW 11.2
MacOS uses clang 14 for x86 and clang 15 for ARM
Linux uses gcc 12.3
Single core:
------------
Tables per Minute - CPU - Cores - compiled or precompiled - OS - metered by
---------------------------------------------------------------------------
0.75 - Ryzen 7700x - 8C/16T - precompiled Linux - Linux (Manjaro) - 200Puls
0.67 - Ryzen 7800x3d - 8C/16T - precompiled Linux - Linux (Chmimera OS) - 200Puls
0.63 - Apple ARM M2 - 4P+4E - precompiled MacOS (ARM) - MacOS 14 - Volker Dirr
0.60 - MacBook Pro M2 - 4P+4E - precompiled MacOS (Intel) - MacOS 15 - yush
0.55 - AMD Ryzen 5900X - 12C/24T - precompiled Linux - mx-linux - PrussianHeathen
0.53 - AMD Ryzen 7730U - 8C/16T - Qt 6.8, gcc 14.2 - openSUSE Tumbleweed - Liviu Lalescu
0.51 - AMD Ryzen 5600G - 6C/12T - Qt 6.5, gcc 11.4 - Kubuntu 22.04 - Volker Dirr
0.46 - AMD Ryzen 3900X - 12C/24T - Qt 6.6, gcc 13.2 - Fedora 38 - HITCHER_I
0.39 - Intel i5 9500 - 6C/6T - precompiled Linux - Debian 12 - PrussianHeathen
0.38 - AMD Ryzen 2700x - 8C/16T - Qt 6.5, gcc 12.3 - openSUSE Leap 15.4 - Liviu Lalescu
0.27 - Intel Pentium G4560 - 2C/4T - precompiled Windows - Windows 10 - Volker Dirr
0.26 - Intel i4460 - 4C/4T - precompiled Windows - Windows 10 - Volker Dirr
0.26 - Broadcom BCM2712 (Pi 5) - 4C/4T - Qt 6.4, gcc 12.2 - Raspberry OS Bookworm, Volker Dirr
0.22 - Intel i5-4308U - 2C/4T - precompiled MacOS (x86) - MacOS 12 - Volker Dirr
0.18 - AMD Athlon II X2 270 - 2C - Qt 6.5, gcc 11.4 - Kubuntu 22.04 - Volker Dirr
0.08 - Broadcom BCM2711 (Pi 4) - 4C/4T - Qt 6.4, gcc 12.2 - Raspberry OS Bookworm, Volker Dirr
The precompiled versions use Qt 6.5.3
Windows uses MinGW 11.2
MacOS uses clang 14 for x86 and clang 15 for ARM
Linux uses gcc 12.3
Multi core: (All threads)
-------------------------
Tables per Minute - CPU - Cores - compiled or precompiled - OS - metered by
---------------------------------------------------------------------------
6.71 - Ryzen 7700x - 8C/16T - precompiled Linux - Linux (Manjaro) - 200Puls
6.60 - AMD Ryzen 5900X - 12C/24T - precompiled Linux - mx-linux - PrussianHeathen
6.23 - AMD Ryzen 3900X - 12C/24T - Qt 6.6, gcc 13.2 - Fedora 38 - HITCHER_I
5.93 - Ryzen 7800x3d - 8C/16T - precompiled Linux - Linux (Chmimera OS) - 200Puls
3.84 - Apple ARM M2 - 4P+4E - precompiled MacOS (ARM) - MacOS 14 - Volker Dirr
3.75 - AMD Ryzen 2700x - 8C/16T - Qt 6.5, gcc 12.3, openSUSE Leap 15.4 - Liviu Lalescu
3.67 - AMD Ryzen 5600G - 6C/12T - Qt 6.5, gcc 11.4 - Kubuntu 22.04 - Volker Dirr
3.12 - AMD Ryzen 7730U - 8C/16T - Qt 6.8, gcc 14.2 - openSUSE Tumbleweed - Liviu Lalescu
2.85 - MacBook Pro M2 - 4P+4E - precompiled MacOS (Intel) - MacOS 15 - yush
2.02 - Intel i5 9500 - 6C/6T - precompiled Linux - Debian 12 - PrussianHeathen
0.99 - Intel i4460 - 4C/4T - precompiled Windows - Windows 10 - Volker Dirr
0.98 - Broadcom BCM2712 (Pi 5) - 4C/4T - Qt 6.4, gcc 12.2 - Raspberry OS Bookworm, Volker Dirr
0.65 - Intel Pentium G4560 - 2C/4T - precompiled Windows - Windows 10 - Volker Dirr
0.44 - Intel i5-4308U - 2C/4T - precompiled MacOS (x86) - MacOS 12 - Volker Dirr
0.36 - AMD Athlon II X2 270 - 2C - Qt 6.5, gcc 11.4 - Kubuntu 22.04 - Volker Dirr
0.30 - Broadcom BCM2711 (Pi 4) - 4C/4T - Qt 6.4, gcc 12.2 - Raspberry OS Bookworm, Volker Dirr
The precompiled versions use Qt 6.5.3
Windows uses MinGW 11.2
MacOS uses clang 14 for x86 and clang 15 for ARM
Linux uses gcc 12.3
Multi core: (Only Performance cores. So WITHOUT SMT, HT, and WITHOUT Efficiency cores)
--------------------------------------------------------------------------------------
Tables per Minute - CPU - Cores (used threads) - compiled or precompiled - OS - metered by
------------------------------------------------------------------------------------------
5.70 - AMD Ryzen 5900X - 12C/24T (12)- precompiled Linux - mx-linux - PrussianHeathen
5.52 - Ryzen 7700x - 8C/16T (8)- precompiled Linux - Linux (Manjaro) - 200Puls
5.25 - Ryzen 7800x3d - 8C/16T (8) - precompiled Linux - Linux (Chmimera OS) - 200Puls
4.86 - AMD Ryzen 3900X - 12C/24T (12) - Qt 6.6, gcc 13.2 - Fedora 38 - HITCHER_I
2.90 - AMD Ryzen 5600G - 6C/12T (6) - Qt 6.5, gcc 11.4 - Kubuntu 22.04 - Volker Dirr
2.77 - AMD Ryzen 7730U - 8C/16T (8) - Qt 6.8, gcc 14.2 - openSUSE Tumbleweed - Liviu Lalescu
2.73 - Apple ARM M2 - 4P+4E (4) - precompiled MacOS (ARM) - MacOS 14 - Volker Dirr
2.72 - AMD Ryzen 2700x - 8C/16T (8) - Qt 6.5, gcc 12.3 - openSUSE Leap 15.4, Liviu Lalescu
2.08 - MacBook Pro M2 - 4P+4E (4) - precompiled MacOS (Intel) - MacOS 15 - yush
2.02 - Intel i5 9500 - 6C/6T (6) - precompiled Linux - Debian 12 - PrussianHeathen
0.99 - Intel i4460 - 4C/4T (4) - precompiled Windows - Windows 10 - Volker Dirr
0.98 - Broadcom BCM2712 (Pi 5) - 4C/4T - Qt 6.4, gcc 12.2 - Raspberry OS Bookworm, Volker Dirr
0.44 - Intel Pentium G4560 - 2C/4T (2) - precompiled Windows - Windows 10 - Volker Dirr
0.39 - Intel i5-4308U - 2C/4T (2) - precompiled MacOS (x86) - MacOS 12 - Volker Dirr
0.36 - AMD Athlon II X2 270 - 2C (2) - Qt 6.5, gcc 11.4 - Kubuntu 22.04 - Volker Dirr
0.30 - Broadcom BCM2711 (Pi 4) - 4C (4) - Qt 6.4, gcc 12.2 - Raspberry OS Bookworm, Volker Dirr
I added the Results for Raspberry Pi 4B:
Single Core:
0.08 - Broadcom BCM2711 (Pi 4) - 4C/4T - Qt 6.4, gcc 12.2 - Raspberry OS Bookworm, Volker Dirr
Multi Core:
0.30 - Broadcom BCM2711 (Pi 4) - 4C/4T - Qt 6.4, gcc 12.2 - Raspberry OS Bookworm, Volker Dirr
More detailed about Bookworm vs. Bullseye vs. Buster. And also 32-bit vs 64-bit in my video:
https://www.youtube.com/watch?v=l6bFVfcvUpc
I added the results from user PrussianHeathen from the www.computerbase.de forum.
see: https://www.computerbase.de/forum/threads/wer-kann-auf-einer-aktuellen-400-x-eur-cpu-einen-benchmark-durchfuehren-benchmark-einer-stundenplan-software.2164680/
Thank you!
Danke, PrussianHeathen!
Thank you!
I added the results of the new Raspberry Pi 5 (2.4 GHz) in the benchmark tables.
There will be more information about power consume and other stuff on my YouTube channel soon.
I added the results from user 200Puls from the www.computerbase.de forum.
see: https://www.computerbase.de/forum/threads/wer-kann-auf-einer-aktuellen-400-x-eur-cpu-einen-benchmark-durchfuehren-benchmark-einer-stundenplan-software.2164680/
Thank you!
I added the 2nd results from user 200Puls from the www.computerbase.de forum.
see: https://www.computerbase.de/forum/threads/wer-kann-auf-einer-aktuellen-400-x-eur-cpu-einen-benchmark-durchfuehren-benchmark-einer-stundenplan-software.2164680/
Thank you!
I added the results from user HITCHER_I from the www.computerbase.de forum.
see: https://www.computerbase.de/forum/threads/wer-kann-auf-einer-aktuellen-400-x-eur-cpu-einen-benchmark-durchfuehren-benchmark-einer-stundenplan-software.2164680/
Thank you!
Danke! Thank you!
My ñew Spanish gift ASUS Vivobook, processor AMD Ryzen 7 7730U (8 cores, 16 threads).
Compiled by me on GNU/Linux openSUSE Tumbleweed, GCC 14.2.1, Qt 6.8.0.
In chronological order:
Single:
1) 0.526316
2) 0.528634
3) 0.529412
16 threads:
1) 3.24324
2) 3.1631
3) 3.12364
8 threads:
1) 2.84024
2) 2.7907
3) 2.76923
The multi-threads were decreasing in speed. At first try the processor was warming, then at 2nd and 3rd tries it went cooler.
Volker, I think my desktop Ryzen 7 2700X 16-threads result might be wrong, because it is much better than my new laptop, while the single and 8-threads are better on the laptop.
Thank you. I added your results.
Maybe repeat you benchmarks if you are not sure.
If you do a benchmark with a laptop, then be careful. If you do it with cell only, then many CPUs will run slower. So please plug in your power connector if you do the benchmark.
(I think we should always do the benchmarks with power connector. Or should we do them with cell only?)
An other problem of Laptops might be not so good cooling. So they might slow down the CPU if it is running for a longer time. But it's difficult to care about that in the benchmark. We can just add the number that we measure.
If other guys have also results, please report. I am also interested in the new Mac M4 and Intel and AMD CPUs with 10 or more cores. But also any other CPU.
The laptop was plugged in the electrical plug.
I repeated my desktop, same software as the laptop:
16 threads:
1) 3.79447
2) 3.77953
3) 3.79447
8 threads:
1) 2.89157
2) 2.88288
3) 2.88
1 thread:
1) 0.387097
2) 0.387097
3) 0.387931
Looks like the laptop is much better on single thread, but does not like multi-threading.
Good question. First I thought it might be because of the cache, but both have 16MB and both have the same number of threads.
Looks like the newer gcc and Qt version are slightly faster.
I am not sure. Should I add them both, keep the old only or update to the new only.
I am not sure. You are in charge! :)
I downloaded the MacOS ARM version, but it says the file is damaged. Cannot run it.
Thank you for report.
Hmm... I don't know the bug yet.
Do you have M1, M2, M3 or M4? Which OS do you have (..., Ventura, Sonoma, ...)?
Maybe try the Intel version? (Yes, that will be a bit slower of course. I will try to locate your problem and I will maybe also check how much slower the intel version is on a M2, so that we will know how much slower it is.)
I am on a M2, Sequoia 15.3.1 (24D70).
MacBook Pro M2, Sequoia 15.3, running the Intel version:
8 Threads:
Tables per minute: 2.84585
4 Threads:
Tables per minute: 2.08178
1 Threads:
Tables per minute: 0.603807
Thank you! I will upload your results.
I compiled the ARM version again, but with a newer Qt version.
I wonder a bit about my results, since the single core are much faster now, but the multi threading is slower. Not sure if I can accept the results with that new Qt version in the results table. Maybe I should update all benchmarks to the latest Qt and fet version.
hmm... Or I should download the old Qt version again and compile with the old version. So it is better to compare.
I upload the new version. Please download the ARM version once again. And try again.
Sorry. I don't know how fast you was. Please download once again. (there was a small type in my last link)
Liviu, do you have an idea:
looks like only updating from Qt 6.5 to 6.6 increased the speed, but only on single core.
The M2 speed increased from 0.62 to 0.76. Multi core was the same.
Or do you think i copy and paste the results wrong in the past and 0.76 was always correct. The 0.76 result looks strange to me if i also compare with the other multi core results.
hmm....
Maybe I must check the seed at the end. Maybe that is different, even i don't know why that should happen.
No idea, sorry, Volker.
Downloaded (from the same link on page 1) again, still the same error.
hmm... I will try Qt 6.8 tomorrow.
Does anyone has experience using FET on rented virtual machine? I am going to have a very complicated timetable, probably will take hours on my MacBook Pro M2. Hoping that I can use some virtual machine service to get more processing power.