more categories for splitting year

[liquid]

Based on what I eperience while creating not an easy timetable I would do suggest to enable more categories than 3. Max number of categories I need years to split to is 7. I do it manually at the moment but it is time and mind consuming. I know it might seem unusual however requirements for the timetable are indeed real not theoretical.

[Volker Dirr]

the number of categories is limited, because too many categories produce too many subgroups. Too many subgroups need to much memory and cpu power.
But if you set subgroups to real students names, you will see that you need much less subgroups. so i suggest to use the real subgroups.

For example a secondary school system like a "Gesamtschule" (german school system) need a lot of categories.
1. category: classes 7a, 7b, 7c, 7d
2. category: german course: 7 beginner I, 7 beginner II, 7 advanced I, 7 advanced II
3. category: englisch course: 7 beginner I, 7 beginner II, 7 advanced I, 7 advanced II
4. category: math course: 7 beginner I, 7 beginner II, 7 advanced I, 7 advanced II
5. category: selectable main course: 7 physics, 7 french, 7 biology, 7 politics
6. category: religion course: christian A, christian B, musilm, jewish
7. category: sport course: boys I, boys II, girls I, girls II
...

so even this simple sample will produce 4*4*4*4*4*4*4=16384 subgroups.
but of course that is stupid, because in each class are only around 25 students. so there are only 100 students in that year. so you need only 100 subgroups.
so an automatic division of the year will generate around 163 times slower and need around 163 times more memory then a manualy set one.


you can also check if the last hint in following section is usefull to your situation:
http://www.timetabling.de/manual/FET-manual.en.html#id_12

[Liviu Lalescu]

Based on what I eperience while creating not an easy timetable I would do suggest to enable more categories than 3. Max number of categories I need years to split to is 7. I do it manually at the moment but it is time and mind consuming. I know it might seem unusual however requirements for the timetable are indeed real not theoretical.

It is not unusual, but we advise users, in case they need more than 3 categories, to use tricks. If the number of subgroups stays small, it is possible that the generation is faster and memory consumption is lower (as Volker said).

The tricks are described here on the forum, but I don't know exactly where. I'll make a resume:

1) Adding 4 categories:

"If each year (for instance 9) is divided by at most 3 categories, you can add year 9 and divide it in 3 categories. If a year is divided by 4 categories (for instance, year 9 is divided by: section (a, b, c, d), language (en, fr), religion and boys/girls), you might consider years: 9a, 9b, 9c, 9d, each divided into 3 categories, and divide each year in the dialog. For more than 4 categories, very unlikely case, you will need to manually adjust groups/subgroups."

2) Adding more categories:

Unite all students of the year in a single activity, so this will be a category. All the divisions in this category will have their activities simultaneously.

[Bobby Wise]

Although I manage everything with 3 catagories, 1 extra catagory would make my life far easier.

I do understand that the program would probably work a lot slower but that doesn't bother me, I don't mind waiting for an hour or even more, some of my TTs I just put on generate and go home. Then when I come in the next morning the TT is waiting for me.

The speed is not important to me, but and extra catagory, that would definitely be appreciated.

I would sacrifice the time for the extra catagory anyday.

[Liviu Lalescu]

Although I manage everything with 3 catagories, 1 extra catagory would make my life far easier.

I do understand that the program would probably work a lot slower but that doesn't bother me, I don't mind waiting for an hour or even more, some of my TTs I just put on generate and go home. Then when I come in the next morning the TT is waiting for me.

The speed is not important to me, but and extra catagory, that would definitely be appreciated.

I would sacrifice the time for the extra catagory anyday.

Are you using 3 categories and want 4, or are you using the trick to make 3+1=4 categories and would like 4+1=5?

I am not sure by how much more categories might slow the generation. This must be calculated and simulated, but I admit I did not think too much of these, as they are not essential to the program. The notion of categories is external to the generation algorithm of FET. Anybody can add as many categories, manually adding groups/subgroups, or using another program to generate the .fet file.

It is not a matter of 3 or 4 categories. If I stop at 4, some others will come and ask for 5, then for 6 or even 7. 3 is a reasonable compromise value.

Some timetables might become impossible if using too many categories, with many empty subgroups, I am not sure though. Also, speed might be very slow, again I am not sure. There are timetables requiring hours. Might require too much if using too many categories (again I am not sure).

I actually started making 6 or 7 categories in the past, but Volker advised me of the dangers. People might tend to abuse these categories, when in fact they don't absolutely need them. The generation should be as fast as possible.

The best would be a single student in a subgroup and a menu to put each student in each necessary group. But I cannot do that personally (not my field, not enough time for that). I prefer to keep open the door of adding the FET generation algorithm into other programs, so I don't care too much about the interface. Again, I stress the fact that you can add as many categories as you want, but only by manually adding groups/subgroups or by generating the .fet XML file.

It is not too difficult to make more categories in FET, as a custom version. I could help with advice and/or code.

[liquid]

You are absolutely right. The possible subgroups often exceed the real need. The subgroup is in fact an attribute which describes a student or a group of ones. The relation is, if I'm not wrong, such that you can assign only one attribute to one student but one attribute can describe more than one student. It's obvious that not all attributes will be in use in reality.
In the example given by Volker, there might be nobody that attends to group a, german course for beginners, english course for advanced II, math course for beginners II, biology, muslim and be a girl. So it is crucial to decide which subgroups (attributes) will be in use and which not. It is a good question but I'm not sure whether you can answer it before the school begins, can you?
This leads to a conclusion that FET should somehow help to remove unused subgroups after the first version of timetable is done with full set of attributes. Here in Poland, the first week of school is  the time when the timetables get finetunning so the next version of it should be more optimised.

[Bobby Wise]

In some cases the school layout would be similar to this:

Grade 1 to 12
Home class A-E

Then I the have the first split:

Languages

Then I have 4 fields of study:

Natural Sciences
Social Sicences
Technology
Commerce

Normally I combine the Languages into the Grade & Home Class so instead of spliting 11a and 11b for instance, I split 11aGer and 11aAfr and then the same for 11b and so on. This works fine but it takes a while longer when assigning the Activities to the Student groups, and it does take a bit more concentration.

Up to now I have always managed to come up with a solutions so this is not crictical for me but it would be nice to have.

[liquid]

It is not too difficult to make more categories in FET, as a custom version. I could help with advice and/or code.

I would appreciate some hints how to do this. It might be interesting experiment. I would also like to know how to detect empty groups as it could be helpful in further optimisation of TT.

[Liviu Lalescu]

I would appreciate some hints how to do this. It might be interesting experiment. I would also like to know how to detect empty groups as it could be helpful in further optimisation of TT.

The code is in src/interface/splityearform .h, .cpp, _template.ui. It is not very nicely written. I will send you my files from my hard disk, but I am not at home right now (I will arrive in ~8 hours from now). I remember that I improved the code in those files, and I accepted 6 or 7 categories, I am not sure.

To detect empty SUBgroups (not groups), is the job of the user, FET cannot detect that. Empty subgroups should simply be removed from all groups.

[Liviu Lalescu]

@liquid: I attach to this post my work on 7 categories (maximum 12, 6, 6, 4, 4, 4, 4 divisions) (it was a long time ago). It is not complete, I cannot remember what I did and what not. You can compare by contents with the official FET files. I will help with advice and possibly more code, let me know.

@Bobby Wise: I will think of what you said. Maybe we will also wait for liquid's tests.

[liquid]

Thanks for that. I'll look at the code though I'm not familiar enough with Qt now and the whole program FET. Qt is not difficult to understand as I read some pages on it. Analysing a program of someone else is more difficult but feasible.
Thus I have questions:
1. What structure (class) rules and stores subgroups and in what file is it?
2. What part of program reads .fet file and processes it?

[Liviu Lalescu]

Answers:

1. engine/studentsset .h and .cpp and engine/rules .h and .cpp
2. engine/rules .h and .cpp, function read(...)

[liquid]

I reviewed the code. Tell me whether I'm right or wrong: the number of years, groups and subgroups is not checked while reading the .fet file. Is it checked somewhere else?
If so it would be better to verify the real need of having more categories (subgroups) by generating a part of .fet file. I would like to go through the process this way before doing bigger changes in the code.
If not ... we'll think a bit more.

[Liviu Lalescu]

I reviewed the code. Tell me whether I'm right or wrong: the number of years, groups and subgroups is not checked while reading the .fet file. Is it checked somewhere else?

It is not checked, indeed. The .fet file can be very large. However, to generate, the maximum total number of subgroups should be MAX_TOTAL_SUBGROUPS (30,000), otherwise FET reports that you should increase this constant.

If so it would be better to verify the real need of having more categories (subgroups) by generating a part of .fet file. I would like to go through the process this way before doing bigger changes in the code.
If not ... we'll think a bit more.

I do not understand.

In FET, the only restriction is when generating, the number of total subgroups should be maximum MAX_TOTAL_SUBGROUPS. Dividing matters only indirectly. If you divide a year by say 6 categories, with divisions: 12, 6, 6, 4, 4, 4, the number of subgroups for this year will be the product = 27,648, so, indeed, there might be some problems.

MAX_TOTAL_SUBGROUPS may be of course increased (just do some checking, though I think it can be on 32 bits, it is not only on 16 bits. MAX_ACTIVITIES should be on 16 bits - which can be changed in places where I used qint16).

[liquid]

I generated manually subgroups of 6 categories. I guess it should be about 10000 subgroups. I inserted them into .fet file and tried to read it. Unfortunately, the program stopped after it encountered a subgroup been added already to another group. If you add such a group in the interface FET informs you about it but goes on. While reading .fet file it gave me a message and whatever I pressed, skip rest or next, program ends.
Anyway, it seems to be easier to create manually needed groups and subgroups than to remove unnecessary subgroups out of tens thousands generated automatically.
I think, however, it would be a compromise to allow to create 4 or max 5 categories. After reading the code it is the matter of interface rather than mechanics of FET. Indeed, the programmed interface is hardly scalable but I think interface was not a goal - rather good conception of algorithm.