is it possible to continue optimize the soft constraints after generation?

[xing]

Hello dear Liviu and Volker

I have read the history of FET that you used heiristic algorithms at the first, but the performance was not so satisfied. So the current very good-use FET came up.
I am wondering if we can obtain better timetables if the soft constraints (constraints with Weight_Percentage<100%) can be optimized further after a timetable is generated? Like use Simulated Annealing or other local search methods to fine-turning the timetable.
I always use MinDaysBetweenActivities to make the timetable more tidy, but sometimes I have to remove this constraint to find a available timetable. So I thought can I set the Weight_Percentage of MinDaysBetweenActivities to 90% or smaller value at first, then fine-turning the obtained timetable to reduce the soft constraints violation as far as possible?
Besides, is it a useful function for FET that one can manually adjust the obtained timetable?

[Volker Dirr]

About the last point:
Yes, but with the current design it is sadly not possible to do this by drag and drag.
You can do it, a bit complicated with FET or a bit more handy with TiTiTo.

About the first point:
Yes and no. (I think the answer is more "no" then "yes")

Yes, you might find optimisations at the end. That is true.

But your suggestion is in my opinion "critical", because guys don't think anymore (or at least less) about the timetable. I always use only 100% weight constraint. If a constraint is not possible with 100%, then i think about why it is impossible and reduce it dwon to 0%, because why should i waste time in tring something that is impossible.
don't forget that a weight of 99% doesn't mean that in 99% it is "fine". So if a constraint with 80% weight is impossible, it will be still impossible with 99,9999% weight. You just wait longer to get the same wrong/impossible timetable.

So one variant might be "fine tunig" as soon as it is ready. In my opinion this "fine" tunig" is wrong, since the correct way is to use 100% weight (if it is possible) or 0% weight (if it is impossible).

Also this fine tuning at the end is "wrong", since there might be much better results at other places that local search methods won't find. So in my opinion the correct way is to enter "better" constraints (with 100% weight).

So in practice i set all needed constraints (except max teachers gaps) with 100% only. Generating that need only a short time. As soon as i have got that result i add teachers max gaps with 100% and a value n. Then i reduce that value n step by step. In normal case it solve at my first school for example with value 3 in seconds. Value 2 in minutes and value 1 in hours. I might try value 0, but in fact that is ciritcal, because i must be able to solve a timetable 1 day (since in worst case the is a bug or problem and i need to be able to fix that in 1 day. I can't say "School is out of order today. Please come back next month. i need to fix the timetable first")
So the maybe only step in between for me might be max 0 gaps with a weight less then 100%, but in fact i normaly have no time to do that "opimisation" also, because i spent already too much time to generate a timetable with max 1 gap for the teachers.

[xing]

Thanks for your detailed explanation.

I agree that optimize a soft constraint by local search may not find better results than alternatively tring different 100% constraints. That's a good tip for me.

So in practice in FET the weight either be 0% (not set the constraint) or 100% (set), what the weight between 0~100 will bring to us?

Of cause the correct way is to adding (reducing) constraints with 100% weight until a timetable cannot (can) be generated. Then, we at least have a workable timetable.
But finding proper constraint setting costs time. If you are lucky, you get the timetable with teachers max gaps with 100% and a value 3 in seconds. But maybe hours later, you realize it is impossible even with value 3 and a bigger value than 3 is nonsense for teachers. Then, a possible way is to set A teacher max gaps one by one for every teacher, but the problem are it is time-comsuming and which sub-group teachers should be selected.

In this case, maybe we can still set teachers max gaps with value 3 and a weight between 0 and 100, then minimize the fitness which is calculated as weighted sum of number of broken rules as in the source code. A perfet timetable is still impossible. But if we need to release timetables a day later, we just let the algorithm run and optimize obtained timetables for a day as far as possible. Maybe 80% teachers' max gaps with value 3 will be satisfied, as a bonus. We lose nothing by doing this.

The above suggestion may not practically useful. I just want to hear opinions.

The FET is a very good to use tool I have ever used, and the core idea of recursion is such a "beautiful" implementation.

[Volker Dirr]

Weights with >0% and <100% are in my opinion only "good" if
... this is the very very last optimision you want to do for a single or a few constraints
... if you want to get results "fast" without optimising your timetable too much
... if you don't know/understand why 100% constraint fails, but you still want it

Optimising constraints by increasing them from 95% to 96% to 97% to .... are in my opinion waisting time. Increasing them to 100% and if it fails seach and understand the bug need less times in most situations. You should try that first if you want to safe time.

From practical:
I done this "100% weight" rules several years at my first school. I was always able to solve with max 2 gaps per teacher in some minutes. (And i/we done a lot of different timetables at that school (FET was used at that school around 8 years. Sadly the school is closed now :-( )

At my second school the dataset was different (i done the tables there for around 5 years). I was (nearly) never able to solve with max 2 gaps for all teachers. With max 4-5 gaps per week per teacher it was as difficult as 2 gaps at the other school. But i was always able to solve with max 3 gaps for full time teachers and max 2 gaps for halfe time teachers. But that already took 1 day to get a result. Of course i might optimise more, but generating more then 1 day is in my opinion not practical.
Only while "corona pandemic" the lessons where shorten and the dataset was suddently much easier, even if only a few lessons were removed.

If you want to do it, then do it. You can use weight with any value that you think is fine. I just don't recommend it.
At least i don't recommend if you have got a lot of constraints with less then 100%. If you only have a few, then i think it is ok to do try it as LAST optimisation.

But as i already said: don't forget that in worst case the result might not be better.
So for example if you solved a timetable with 3 gaps max for all teachers (100%), then maybe 10% already have got only 2 gaps per week.
If you now add a second constraint with 95% weight and max 2 gaps per week, then it doesn't mean that 95% of the teacher will get 2 gaps per week! This will happen only in best case (so with an "easy" dataset. But if the dataset is easy, why not use 100% from the start?) In worst case you might get again only 10% with 2 gaps per week (even with 95% weight!). But i can tell you, that you will wait (much) longer to get a result.

[xing]

Thank you so much for the replay from practical view.

Learned a lot.