After reading Liviu's post, I would like to explain my perspective on FET development.
1) I think that implementing a multi-core version of FET would improve significantly the performance of timetable generation. Most modern computers have 4 or more cores. So, a multi-core version of FET would improve its performance. For example an 8 core CPU would solve the timetable in around 1/10 of the time.
2) About integrating (uniting) all FET variants into a single version: It seems really interesting but it might confuse the inexperienced user. On the other hand, FET would need a single executable.
3) About getting rid of the limits seems to be a very good generalization on FET's usage.
4) I think that FET works very well for solving timetables which do not need to be changed/modified during the school year. In my country (Greece), most of the time teachers are hired "incrementally". So, most Greek schools need to change/modify their timetables around 10-15 times even up until April! This means that one needs to be able to easily change/modify an already existing timetable. Under this perspective, a new feature of adding/removing/modifying constraints on a existing timetable (in addition to locking/unlocking activities) would be highly useful. What about other countries? Do you need to modify existing (FET generated) timetables during the school/academic year or not?
5) Sometimes, I needed to add identical constraints for the vast majority of teachers (for example max hours continuously = 4), while at the same time a couple of teachers were an exception. I think that a new feature for adding constraints for (predefined) groups of teachers/students/rooms would be highly useful.
6) The constraints teachers/students/rooms not available times sometimes cause problems. I think that adding a button for displaying a report for all teachers / all students / all rooms not available times would simplify the monitoring of these constraints. Especially, I think that for rooms not available times with a weight <100, displaying the weight percentage in each time slot would be very useful.
Vangelis.
Thank you, Vangelis!
About (1): It is impossible to generate a single timetable on multiple cores. I had in mind a multiple generation, so you can generate 8 timetables in parallel.
About multi-core:
1) What is the (practical) difference between generating 8 timetables in parallel and running 8 instances of FET simultaneously?
2) Why is it impossible to generate a single timetable on multiple cores? Isn't it possible to modify the code so that FET generates a single timetable utilizing all 8 cores?
(1) None, but single interface would be easier.
(2) It seems impossible to me. The algorithm is unfortunately linear.
As someone who uses FET and FET-BP, the possibility of combining them into a single version is very appealing, so I'm in favour of this idea. It would be nice (but a bigger job) to be able to export FET-BP timetables in a CSV format that could then be imported directly into the "regular" FET version. For example, to be able to export from FET-BP an activities CSV file with the "Days per week" data exported in the teachers column, and the Hour data exported as an activity tag (Block A, etc.). (Of course, these files would still need some manual work before import, with the split duration information adjusted, etc.)
I would also like the possibility of exporting/importing constraints. For example, sometimes I start a new year with a "fresh" FET file, and then import teacher lists, etc., from CSV files (usually exported from the previous year's FET file). Most teachers also have a homeroom (that typically doesn't change from year to year), but it isn't possible to export/import "A teacher has a home room" space constraints, so all of that information has to be re-entered. (Also, I sometimes have to share things like the teacher home room information with others, and so even just an export option for these constraints would be helpful.)
A few more Locking/Unlocking options would be helpful too. For example, "Lock all activities of a specified year group", "Lock all activities of a specified teacher", etc. (I usually accomplish this with activity tags, but that has sometimes required me to create a new tag and then tag all the relevant activities in order to get this done, which can be time consuming.)
Quote from: Darren McDonald on March 20, 2021, 06:34:54 PM
As someone who uses FET and FET-BP, the possibility of combining them into a single version is very appealing, so I'm in favour of this idea. It would be nice (but a bigger job) to be able to export FET-BP timetables in a CSV format that could then be imported directly into the "regular" FET version. For example, to be able to export from FET-BP an activities CSV file with the "Days per week" data exported in the teachers column, and the Hour data exported as an activity tag (Block A, etc.). (Of course, these files would still need some manual work before import, with the split duration information adjusted, etc.)
I got rid of the limits, and now I began combining MA into official, which is terrifically hard. I am so tired :) . I don't know if I'll manage. But if I do, please keep this idea and remind me when I combine BP into the official (which should be easy).
Quote
I would also like the possibility of exporting/importing constraints. For example, sometimes I start a new year with a "fresh" FET file, and then import teacher lists, etc., from CSV files (usually exported from the previous year's FET file). Most teachers also have a homeroom (that typically doesn't change from year to year), but it isn't possible to export/import "A teacher has a home room" space constraints, so all of that information has to be re-entered. (Also, I sometimes have to share things like the teacher home room information with others, and so even just an export option for these constraints would be helpful.)
Did you see the XML .fet text for this? You might be able to create it with scripts.
Quote
A few more Locking/Unlocking options would be helpful too. For example, "Lock all activities of a specified year group", "Lock all activities of a specified teacher", etc. (I usually accomplish this with activity tags, but that has sometimes required me to create a new tag and then tag all the relevant activities in order to get this done, which can be time consuming.)
I believe this is achievable now. Go to the timetable time horizontal view and select one or more rows, "Lock" or "Unlock".
Quote from: Liviu Lalescu on March 20, 2021, 06:46:07 PM
Did you see the XML .fet text for this? You might be able to create it with scripts.
Yes, I may look into this for the coming school year (although I'll probably just start from the current FET file and make modifications directly through the FET interface this time).
Quote from: Liviu Lalescu on March 20, 2021, 06:46:07 PM
I believe this is achievable now. Go to the timetable time horizontal view and select one or more rows, "Lock" or "Unlock".
Great tip! Thanks!
Quote from: Vangelis Karafillidis on March 19, 2021, 08:39:11 PM
What about other countries? Do you need to modify existing (FET generated) timetables during the school/academic year or not?
Usually I modify timetable twice during year (1 and 2 semester). But, in this year I have 6 (six!) official timetables. Unfortunately, places of the same activities are wery different, and, as a result, I have different timetables. (Russia, Crimea)
Quote3) Make multiple generation start on n threads, where the user can choose n between 1 and the number of cores of the computer.
In the summer I will have free time and I will be able to test the old and new FET on my timetable files.
Thank you! :)
I added a post: https://lalescu.ro/liviu/fet/forum/index.php?topic=4794.msg26575#msg26575 . It is unfortunately an admin section there, so reply here, in this topic.
I hope you'll like my work :)
I added a new snapshot version, fixing a bug: the generated locked file was saved with mode "official" always. I changed the location of the snapshot (test) version: https://lalescu.ro/liviu/fet/download/test/
In this tarball in file src/engine/generate.cpp:6210:20 typing error: "smhd" was not declarated in this scope, and before (in line 6204) this variable is "shmd". (h <-> m)
Thank you, Alex! Corrected, I added a new snapshot, only the tar.bz2 for now (keeping also the previous snapshot). The usual link now: https://lalescu.ro/liviu/fet/download/test/ , in the other directory (Backup-fet) on my server I prefer to keep personal snapshots, not public ones.
I compiled correctly on my computer because I use Qt 5.15.2 and the #ifdefs are taking a favorable turn. You are using an older Qt, and the #ifdefs pointed out this mistake of mine. If should be smhd (h in smhd means "half", d means "day").
Before the official release I will make sure to compile with Qt 4 to check errors like this.
You're right. I use older version Qt 5.11.3 (GCC 8.3.0, 64 bit). How and where I'll can know about new versions (snapshot) for testing them?
Hmm... I will announce here new more important snapshots, then in the future on the forum board snapshots: https://lalescu.ro/liviu/fet/forum/index.php?board=15.0 . But on the snapshots board I will announce a snapshot after I consider the snapshot reached a good form.
We'll keep in touch - maybe you have a better method to suggest to me.
OK, I am (nearly) ready with (3) :) I hope you'll like it as much as I do. Especially if you have multi-core computer. Enter generate multiple dialog.
I added a new version: https://lalescu.ro/liviu/fet/download/test/
Please let me know (few reactions lately :( ).
Quote from: Liviu Lalescu on March 31, 2021, 07:09:38 PM
OK, I am (nearly) ready with (3) :) I hope you'll like it as much as I do. Especially if you have multi-core computer. Enter generate multiple dialog.
I added a new version: https://lalescu.ro/liviu/fet/download/test/
Please let me know (few reactions lately :( ).
Sorry, I was waiting for new version. I'm interested to try testing multiple threads for generation, because usually create 3 versions of timetable for considered by school administrator (for claims one of them)
Thanks for you job! It's very useful to me personally and my school as a whole. I use FET from 2016 (archive of our school timetables http://pochtschool.ho.ua/tt.php (http://pochtschool.ho.ua/tt.php))
Now a new version are compille... (about 40-50 min as ususally). I send some report later.
Sorry, for my English, I hope you understand me.
Thank you! I understand :)
For compilation: if you have more cores (like 4), write "make -j 4". I have 8 cores/16 threads and write "make -j 16" and it compiles in ~4.5 minutes (Ryzen 2700x).
Thanks! This is really 3x faster:
time make (1 core)
real 64m17,696s
user 59m27,125s
sys 4m49,418s
time make -j 8 (4 core x 2 threads)
real 18m59,422s
user 67m33,182s (??? what is it mean?)
sys 4m30,277s
AMD Ryzen 3 3200U (4 CPU 2 Threads per core)
67 minutes I think is total time, summed for each thread.
I am ready with what I had in mind, and I added a new version: https://lalescu.ro/liviu/fet/download/test/ . This fixes crashes in generate multiple if using the new Qt 6 and interrupting the generation when no complete timetable was generated.
I am very happy with the results and it won't take long until the official FET-6.0.0 release :)
Before official release, bugs in FET-6.0.0 "It's not a bug" (c) ;)
I create new topic with testing results in "ReportTest Results" https://lalescu.ro/liviu/fet/forum/index.php?topic=4798.0 (https://lalescu.ro/liviu/fet/forum/index.php?topic=4798.0)
Quote from: Liviu Lalescu on April 01, 2021, 04:39:11 PM
I am ready with what I had in mind, and I added a new version: https://lalescu.ro/liviu/fet/download/test/
I'm getting an error when compiling on Mac with Qt 6.0.3. (Let me know what info I can provide, but it looks like the most relevant error message is "fatal error: 'sys/errno.h' file not found".
Hmm... please attach here the whole log, if possible. I googled the message you wrote ("fatal... found"), but it is not on the internet.
Quote from: Liviu Lalescu on April 02, 2021, 07:07:44 AM
Hmm... please attach here the whole log, if possible. I googled the message you wrote ("fatal... found"), but it is not on the internet.
I've just compiled the latest snapshot without a problem, so whatever the issue was, it's now fixed!
Thank you, Darren!
Hello dear sir @Liviu Lalescu
It is a great honor for me to see you again as you make this mighty extraordinary feat.
I just noticed, while working on the new version, a problem using multiple threads, on a Linux system, as the app sometimes stops with the appearance of a message: "segmentation fault (core dumped) ./fet"
And sometimes it works normally on the same file and the same options.
Thank you, Mr. ghani1990,
Please call me Liviu :)
Hmm, this is a very very bad thing! :'( Please tell me the exact version of FET you are using (is it an old snapshot, before 13 April), do you compile it or use the precompiled version from my site, what GNU/Linux system. How many threads do you use, and what is the maximum of your processor?
Also, could you send me your file? (I'll keep it private.)
Quote from: Liviu Lalescu on April 22, 2021, 02:35:36 PM
Thank you, Mr. ghani1990,
Please call me Liviu :)
Hmm, this is a very very bad thing! :'( Please tell me the exact version of FET you are using (is it an old snapshot, before 13 April), do you compile it or use the precompiled version from my site, what GNU/Linux system. How many threads do you use, and what is the maximum of your processor?
Also, could you send me your file? (I'll keep it private.)
Hi my friend Liviu.
Please call me ghani :)
I am using the latest version of your site 6.0.1 under the Linux Manjaro system, my computer has the capacity of 4 threads, and I used all four for one schedule, sometimes the process fails at the beginning completely and fet closes, and sometimes the work continues normally.
By the way, if the operation is complete, the result is surprisingly fast.
Thank you.
The file is in attachments ;)
Hello, ghani :)
Please confirm:
1) My precompiled GNU/Linux executable, not compiled by you. Please go to Help/About Qt and tell me the Qt version.
2) When you begin generation multiple, it crashes at start? After less than 1 second? After 1-5 seconds?
Quote from: Liviu Lalescu on April 22, 2021, 02:56:28 PM
Hello, ghani :)
Please confirm:
1) My precompiled GNU/Linux executable, not compiled by you. Please go to Help/About Qt and tell me the Qt version.
2) When you begin generation multiple, it crashes at start? After less than 1 second? After 1-5 seconds?
1- yes it yours, Qt version 5.15.2.
2- The behavior is different every time, generally it crash immediately, or in 3 seconds in total.
Quote from: ghani1990 on April 22, 2021, 02:18:56 PM
Hello dear sir @Liviu Lalescu
It is a great honor for me to see you again as you make this mighty extraordinary feat.
I just noticed, while working on the new version, a problem using multiple threads, on a Linux system, as the app sometimes stops with the appearance of a message: "segmentation fault (core dumped) ./fet"
And sometimes it works normally on the same file and the same options.
I confirm the bug, on a laptop with openSUSE :'(
The only advice I can give you: compile with Qt 6.0.3.
I'll try to isolate the bug, but it is very hard.
Really, it's a tough issue, can you confirm that it does not exist with version 6?
Well, I will try to build the new version with QT 6.
What if you release the new version, with QT 6? Can't you??
I isolated some random seeds which crash on my computer with Qt 5, but I never crashed with Qt 6.
I cannot release with Qt 6, because linuxdeployqt only works with Qt 5.
Also, I need to take care of this very ugly bug. It must work on any Qt.
I would like to tell you that I would like you to wait for a possible fix from me, so that you can test it.
Well, I am waiting for the solution, and I will test all versions until we get a good result.
I'm very grateful to you for all your help and for the wonderful work you are doing, my friend
Working on it... bug isolated, see attached screenshot for my work. Bug maybe in generate.cpp, line 6811.
:o
I am sure that the solution will be very soon, with your insistence.
Waiting for that.
It seems that it is a Qt-5 problem, from Qt-5.14.0 to Qt-5.15.2.
I will investigate further the Qt code, but I think I cannot make a hack to fix FET behavior on Qt 5.14.0 to Qt-5.15.2.
I could compile for you with Qt-5.12.10, if you really need, but I would advise you to get Qt-6 and compile, it is not difficult at all. Please let me know. I can help you in both variants, either way you choose.
Please, Don't tire yourself out, my friend.
I will build it on manjaro with Qt 6 and tell you the result.
Install gcc and g++ (or gcc-c++), I have 10.3.0, but maybe you can only install older.
Make a Qt account (free), use the online installer, install Qt 6.0.3 gcc 64-bit. qt.io is the site, you need to dig a bit for the link. Let me know if you need help.
"~/Qt/6.0.3/gcc_64/bin/qmake", then "time make -j 4" (time if you want to know how long it compiled).
I tried something and it seems I found a solution even for Qt 5.15.2. I hope you can help me with testing. I will try to put a test version for you soon.
Dear ghani,
I hope I found the best solution. I have put a snapshot for GNU/Linux exe: https://lalescu.ro/liviu/Backup-fet/fet-6.0.2-snapshot-23-apr-2021-00_01-gnu-linux-executable.tar.bz2 (the best is to go to https://lalescu.ro/liviu/Backup-fet/ and get the one named with "GNU/Linux exe", I might change something)
Please let me know if this version is OK and does not crash anymore. It is very important.
Thank you for your help!
Sorry for the delay due to my internet speed, and also my old computer. It took a full hour and three minutes to build.
I can confirm that the problem disappeared in the version that I built on Qt 6.
I will try the new version that you uploaded and tell you the result.
Thank you!
I am so sorry for your time! I have a very good internet connection and forget about this when talking to other people!
If you have more patience please try, as you say, the snapshot gnu-linux-exe.tar.bz2 (precompiled with Qt 5.15.2 by me) and let me know if the bug is gone. It is important for the people who have Qt 5.15.2 or who want to download my precompiled version.
Don't be sorry my friend, you are doing an amazing job and we have a duty to help as we can.
I am currently doing different tests on multiple files, so far the problem has gone, it's working perfectly, I will run more tests and tell you.
Thank you for your precious time
Thank you for your nice words, ghani! :)
I might go to sleep now (01:13 here :) )
Well my friend, good night and sweet dreams.
Tomorrow we will complete the qt investigations ;D
After several tests on the file that i sent above, with the new version, the execution time with 4 threads takes a long time (12 minutes) more than the normal execution (7 minutes)!?
Could this happen normally?
Practically I did not encounter the crash issue again.
I think the issue is solved.
Only I have a suggestion in the case of multiple timetables why I should delete the output folder every time, why the files are not output in new folders with the same name with a number's suffix , 1,2,3 etc.
I think I understood the reason for the delay in the case of a single schedule, on N threads.
Fet does not launch parallel execution. Rather, it launches the same file to execute in N threads, and the one that ends the first is the one that is taken into consideration. Correct me if I am wrong?
In the case of multiple generation of N timetables, of course, it waits for each thread to end on its own and then gives all the results!
Thank you for your help!
Suppose you start 2 generations on 4 threads: all 4 threads work on the first solution. When it is found, it is written and then the 3 threads (which did not finish) continue the generation and the 4th (which finished) starts a new generation. Which of these finishes first is the second timetable. And similarly for other number of timetables.
It is best to need to delete the directory manually, I think. Because the user normally generates many timetables at once.
Thank you for the explanation and for your hard work.
I suggest adding settings to choose the default mode of operation.
Meaning instead of choosing the The mode of operation every time through the menu New, just I go once to settings and i set my default option.
Why should this preferance exist? Because the Custom FET versions are basically each one belonging to a specific country or group, so I think it is better for me to choose the mode once in the settings instead of choosing every time I create a new file!
Thus, your work is wonderful in the endeavor to collect all copies in one copy, and at the same time it is easy to use the program as if it is directed to a specific mode.
Of course, these settings are optional, they can be edited by the user, or they can be left as they are now.
What do you think?
You are welcome, ghani, and thank you for your suggestions!
I inform you that I might release FET-6.0.2 tomorrow afternoon or on Monday afternoon the new version. You contributed to it being better!
About your suggestions (including the one with not manually having to delete directories): I will add them in the TODO list, with your name. I wrote why I do not like the first one, sorry; about the second one (default chosen mode for the file): it is a very appealing suggestion, but I still like it more as it is now. The new file is a function used very rarely, only once in the beginning, so it is not really a waste of time. Another reason: maybe I add more modes in the future, and the user does not know that.
Hello dear Liviu.
Thank you very much for the compliment, as usual you are doing an amazing job and my participation it is just a small drop in the ocean of your great work.
I understand you very well especially at the point that you may add new modes in the future, it really should be in the user interface.
Also for the first suggestion you are right, in case many tables are desired you can put the number of tables once and wait ;)
Thank you, ghani! :)
Dear ghani,
I just released the new version, 6.0.2, with your reported bug fixed. Thank you for the collaboration! :)
congratulations dear Liviu,
Today, I downloaded the new version, and I also built it , for theme's integration in linux with GNOME environment.
You surprised me, dear, with my name in the "thanks to" window, this is a great honour for me. :-[ :-[
Thank you very much for your amazing work.
Thank you, dear ghani! :)
You are welcome!