FET Forum

FET Development => Suggestions => Topic started by: kdsayang on April 07, 2008, 12:47:19 PM

Title: import and export files
Post by: kdsayang on April 07, 2008, 12:47:19 PM
hi...

For some user, inputting all the subjects and subject code, teacher etc is a tedious work... So, I suggest that FET can import file to ease the user from inputting the details... We can save 1/3 or 1/2 time just to input that... I think the most suitable format is excel file format. Then the user can concentrate on inputting the activities and constraints etc

So, for the export files...  I think excel file format is the most suitable file because normally, me or other user (I think) edit the *.html files from FET in excel and also print it there... There are so many things that u can manipulate the data in excel.

to Volker Dirr: Hi, I've been to your website and download "Get unofficial FET 5.5.2.alpha.vd1 Windows executable (with CSV import)". When I extract It, there is no application *.exe file... So, do I have to compile the files to generate the *.exe file???

TQ
Title: Re: import and export files
Post by: Volker Dirr on April 07, 2008, 01:36:23 PM
Ohh.. sorry. You are right. The exe is missing.
Yes, so you must compiler yourself :-(
You can import only teachers, rooms, subject and years.

I am currently coding a better import filter, so you can also import groups, subgroups, buildings, ... . The internal function already work well, but i am a beginner with Qt. So the user interface is still bad.
That is why i will not compile the old version again. Please do it yourself.
I am working on the new version and i will release that as soon as possible.
but like i told: i am a Qt beginner and i must also work as teacher, so it may take one or two weeks.
Title: Re: import and export files
Post by: Liviu Lalescu on April 07, 2008, 02:10:10 PM
Did you open a .fet file with a text or xml editor? It is easy to understand. Please do that to see it and maybe you can find a way to generate a .fet file from an excel data.

I have to admit I did not think about ways to import/export data, because I tried to concentrate on the algorithm. But I hope Volker and maybe other contributors will work on that.
Title: Re: import and export files
Post by: kdsayang on April 07, 2008, 03:14:51 PM
Quote
but like i told: i am a Qt beginner and i must also work as teacher, so it may take one or two weeks.

hehe... I also work as a teacher.... I've zero knowledge of Qt. Maybe it take months for me to compile and publish it.. hehe... Just do your best.. Slow and steady
Title: Re: import and export files
Post by: kdsayang on April 07, 2008, 03:19:14 PM
QuoteDid you open a .fet file with a text or xml editor? It is easy to understand. Please do that to see it and maybe you can find a way to generate a .fet file from an excel data.

Yes I did.... The files are stored in XML syles... I think. hehe.. I'll try to do that.

Quote
I have to admit I did not think about ways to import/export data, because I tried to concentrate on the algorithm. But I hope Volker and maybe other contributors will work on that.

I also hope Volker work on that... Ganbatte Volker
Title: Re: import and export files
Post by: kdsayang on April 07, 2008, 03:29:56 PM
For suggestion...

for the input file, we can separated it to 5 DIFFERENT files... I suggest the file extension is *.FIF=FET Input Files
(i) course list
(ii) room list
(iii) lecturer list
(iv) days per week
(v) period per day


for course list --> (i)

Year/Course, Group/Course group, Subgroup, subject code, subject name, number of student (-1 for auto), split into, duration, if activites on the same day force consecutives

*DIT,**DIT 1,MTA 223,***-0, Basic Computer Hardware,****-1, 1,3,
*DIT,**DIT 1,MTA 223,***-0, Basic Computer Hardware LAB,****-1, 1,3
*DIT,**DIT 2,MTP 113,Group 1, Basic Programming,35, 1,4
*DIT,**DIT 2,MTP 113,Group 1, Basic Programming LAB,35, 1,4
*DIT,**DIT 2,MPP 223,***-0,Oganization Behavior,30, 2,4,4,1 <-- why extra 4 and 1?? Becoz this 4 for another class period and 1=enable, -0=disable activites on the same day, force consecutives

for room list --> (ii)

room name, building, capacity,
A-A1,Kuliyah Muamalat,40
A-A2,Kuliyah Muamalat,40
B-A1,Kuliyah Arab,40
Computer Lab 1, Kuliyah IT,30

for lecturer list --> (iii)

lecturer
Hanif Ali
Liviu Lalescu
Volker Dirr

for days per week --> (iv)

days per week, day 1,day 2,etc
5,Sunday,Monday,Tuesday,Wednesday,Thursday

period per day --> (v)

period, hour
23,08:00,08:30,09:00,09:30 etc


Legend
*     DIT= Diploma In Information Technology
**    DIT 1= Diploma In Information Technology in semester 1
      DIT 2= Diploma In Information Technology in semester 2      
***   -0= A value to show that there is no subgroup. If there any, just type the subgroup name
****  -1= automatic computing. If u don't want to use -1, just put the total of students

Finaly, we can add the ACTIVITIES, TIME CONSTRAINTS & SPACE CONSTRAINTS by manually.

I think it will be easier if we use the text documents *.txt as the input file as long FET can understand it. But for the long run, *.csv sytles in text documents or *.xls is better.
Title: Re: import and export files
Post by: Volker Dirr on April 07, 2008, 06:43:40 PM
I remember there was an other solution some time ago on the mailling list. Here it is:
Quote
If i understand correct, then you can use it for example to import
teachers name from csv to a fet data file.

1. you need a csv file with teachernames (no seperator, just new lines)
example:
Tom
Al
Fred

2. visit http://www.creativyst.com/cgi-bin/Prod/15/eg/csv2xml.pl

3. Write into "Element ids" this: Name
Write into "Input" the csv teacher names.
Write into "Doc_id": Teachers_List
Write into "Row_id": Teacher

4. press convert


About your suggestions:
Quote
for course list --> (i)
Year/Course, Group/Course group, Subgroup, subject code, subject name, number of student (-1 for auto), split into, duration, if activites on the same day force consecutives
Currently i am working only on Years; Groups;subgroups.

Looks like you also want to import the whole lesson planing. My source is very portable. So i can try to add that later. But i will do it different. split into is not a good suggestion. Example: someone write duration 5 and split into 2 lessons. So what should i do? That's why i prefer "duration" and "splitted duraion" and so you can say: 5; 4+1 or 5; 3+2 or 5; 2+2+1 or ...

Quote
room name, building, capacity,

i am working on that. (Just a little bit more common, you will see hopefully in a week.)

Quote
for lecturer list --> (iii)

i am working excactly on that.

Quote
for days per week --> (iv)
period per day --> (v)

hmmm.... that's a little bit different from my current csv import style.

but i think that can be added later.

currently i try to add the import filter very common, so everyone should be able to use it with his current school administration software. Wait a little bit and you will see that.
The last two suggestions are pretty special and doesn't fit to files of school administration software that i know.
Title: Re: import and export files
Post by: kdsayang on April 07, 2008, 10:27:22 PM
Quote
Example: someone write duration 5 and split into 2 lessons. So what should i do? That's why i prefer "duration" and "splitted duraion" and so you can say: 5; 4+1 or 5; 3+2 or 5; 2+2+1 or ...

If someone want 5 duration and split into 2 lesson, you meant that the class have 2 activites per week and each class have 5 period right? or this "4+1 or 5; 3+2 or 5; 2+2+1" mean anything else??
-->4+1 = 2 activities per week. 1st activity is 4 period. 2nd is 1 period
-->2+2+1 = 3 activities per week. 1st activity is 2 period. 2nd is 2 period. 3rd is 1 period

Let say that the teacher have 4 activites per week and each period is 2+1+2+1.
The input will be:

Year/Course, Group/Course group, Subgroup, subject code, subject name, number of student (-1 for auto), split into, duration, if activites on the same day force consecutives

*DIT,**DIT 2,MPP 223,***-0,Oganization Behavior,30,4,2,1,2,1,1

Quote
hmmm.... that's a little bit different from my current csv import style.

but i think that can be added later.

may be you add it like this... Instead of doing it horizontal, u make it vertical???

for days per week --> (iv)

horizontal
days per week, day 1,day 2,etc
5,Sunday,Monday,Tuesday,Wednesday,Thursday

vertical
5
Sunday
Monday
Tuesday
Wednesday
Thursday

period per day --> (v)

horizontal
23,08:00,08:30,09:00,09:30 etc


vertical
23
08:00
08:30
09:00
09:30
etc

Quote
The last two suggestions are pretty special and doesn't fit to files of school administration software that i know.

Maybe it doesn't fit to fit to files of school administration software that u know. But it fit me.. hehe and now u know hehe  ;)
Title: Re: import and export files
Post by: Volker Dirr on April 26, 2008, 01:56:43 PM
sorry for delay.

QuoteThe input will be:

Year/Course, Group/Course group, Subgroup, subject code, subject name, number of student (-1 for auto), split into, duration, if activites on the same day force consecutives

*DIT,**DIT 2,MPP 223,***-0,Oganization Behavior,30,4,2,1,2,1,1

i did it a little bit different. just open your current data set and export it. then you can see how to import activities.

Quotemay be you add it like this... Instead of doing it horizontal, u make it vertical???

you are right. that fit to my style.
but i haven't enought time now to do that.

please check the current beta ( http://www.timetabling.de/ ) and tell me if it work well. i released source and windows executable. i still need to do a few code cleanups, but i haven't enough time at the moment and i think the current version is good enough to release it as a beta version.

i will maybe add import of days and periods later.
Title: Re: import and export files
Post by: kdsayang on May 04, 2008, 11:39:45 AM
Quotesorry for delay.


please check the current beta ( http://www.timetabling.de/ ) and tell me if it work well. i released source and windows executable. i still need to do a few code cleanups, but i haven't enough time at the moment and i think the current version is good enough to release it as a beta version.

i will maybe add import of days and periods later.

hi,

I've tried this new features of yours. It is very good. In my opinion, I think It is better your put one subject per column in *.csv instead of cramming all of them in just one column. It look nicer and easy to troubleshoot. Well done. Love it...  ;D

Can u PLEASE update it to the latest versions?? TQ

Image BELOW is the old one.... Image ABOVE is my suggestion...

Title: Re: import and export files
Post by: Volker Dirr on May 04, 2008, 12:33:10 PM
QuoteIn my opinion, I think It is better your put one subject per column in *.csv instead of cramming all of them in just one column. It look nicer and easy to troubleshoot. [...] Image BELOW is the old one.... Image ABOVE is my suggestion...

hmmm.. i don't understand what you mean.
can you explain it one more time please?

did you already notice/checked, that several fields are allowed to be empty?

each line contain one activity. to each line must exacly contain one subject.
it can contain (but must not) a teacher set, a student set, activity tag or room constraint.

it must have value total duration or splitted duration. if you enter both values, then both values must be the same.

QuoteCan u PLEASE update it to the latest versions??

not today, hopefully next weekend.
Title: Re: import and export files
Post by: Volker Dirr on May 04, 2008, 01:53:18 PM
Now i understand what you mean. You didn't mean "subject", you mean "activity".

ok. i will explain. (But maybe the english terms are incorrect, because i have installed only a german version and maybe i translate incorrect.)

if you want to import a cvs-file with openOffice calc, then you need to do this:
- open cvs file
- check separators "others" and add sign | in that field.
- remove sign " from textseparator
- press ok.

if you want to export a cvs-file with open Office calc, then you need to do this:
- save cvs file, check "filter settings" and press "save"
- modify field separator to sign |
- modify text separator to no sign
- press ok.


hmmm... maybe i should change sign | to sign ;
and add textseparators.
advantage: nobody need to change settings in openOffice
disadvantage: maybe problems if someone use sign ; or " in teacher/building/students/subjects/... names.

Title: Re: import and export files
Post by: Volker Dirr on May 04, 2008, 04:06:34 PM
maybe it is better to use other field separators and also use text separators.
i just checked it with one openOffice version.

but i want to be more sure that other software do it similar.

so can everybody please tell be the used field and text separator of his calculator?
(ok course i will code it common, but i want to have good default settings.)

if you don't know your field and text separator, then just open a spread sheet and enter a 2 rows and to columns. just add 2 words and 2 numbers, save that file as csv and sent it to me.
please also tell the software you are using.
Title: Re: import and export files
Post by: kdsayang on May 05, 2008, 01:48:52 AM
I'm using M$ Office 2007. It works. U have to change the 1000 Separator from , to |. U can change that in Regional and Language Option in Control Panel (Windows XP)

Then in Regional Options TAB, click customize. Change , in List separator drop down menu to |. Then open the *csv file and walla... every activities on their own column.. hehe.. TQ

Below is the ouput
Title: Re: import and export files
Post by: Volker Dirr on May 05, 2008, 01:36:53 PM
ok. i don't have M$ Office 2007.

i just checked openOffice, KSpread and i read an aritikle about csv.
It looks like most use field separator , (comma) and text separator ".

by your answer i know, that MS also use field separator , by default.
but i don't know if they also use text separator " by default.

can you please check that?

just reset your office settings and do this:
write into field A1:  Hallo
write into field A2:  Mr. "genius" Brown
write into field B1:  4
write into field B2:  4.5

save that as cvs and post it here please.
(i want to know what office do if a name (string) is used (A1), if a text separator is used (B2), if a number (int) is used (B1) and what happen if a number (double) is used.

i hope it is similar to other software, so i hope it is also:
"Hallo","Mr. ""genius""Brown"
4,4.5

if MS save it different, please post that here and please also try if you can open previous 2 lines without changing anything in your office settings.

please let me know, i will change default fet output csv.

Maybe other can also check other spreadsheet software?
Title: Re: import and export files
Post by: kdsayang on May 05, 2008, 04:37:48 PM
1st. What I did was not changing anything in M$ Excel 2007. FYI, for changing the separator setting, I just need to change it in Windows in Control Panel --> Regional and Language Options. I don't need to change anything on M$ Excel 2007.

When I don't change anything, the output is the same as the input.
A1:  Hallo
A2:  Mr. "genius" Brown
B1:  4
B2:  4.5

2nd. When I change the setting

QuoteU have to change the 1000 Separator from , to |. U can change that in Regional and Language Option in Control Panel (Windows XP). Then in Regional Options TAB, click customize. Change , in List separator drop down menu to |
It changed to
A1 : Hallo,4
A2 : Mr. "genius" Brown,4.5

Below is the file....
Title: Re: import and export files
Post by: Volker Dirr on May 05, 2008, 06:23:40 PM
Quote1st. What I did was not changing anything in M$ Excel 2007. FYI, for changing the separator setting, I just need to change it in Windows in Control Panel --> Regional and Language Options. I don't need to change anything on M$ Excel 2007.

ok.

Quote
When I don't change anything, the output is the same as the input.
[...]
2nd. When I change the setting
[...]

ok. so windows (and/or excel) also use the same default field and text separators like openOffice.

i will change my source.
changing the export filter will be easy.
changing the import filter will be a more difficult then the previous/current version.

i will do that in the next days.

Title: Re: import and export files
Post by: kdsayang on May 08, 2008, 06:23:35 AM
It seems that *.csv can only save one worksheet in one *.csv file. Unlike *.xls (excel) where we can have many worksheet in only one file.

What I'm trying to say that, in *.csv, we can have 6 different files because *.csv file only support one worksheet.

Unlike *xls where it can save that 6 different files into just one file because it support multiple worksheet.

So, my suggestion that, for future FET release, FET can import and export *.xls file

TQ
Title: Re: import and export files
Post by: Volker Dirr on May 08, 2008, 05:06:55 PM
QuoteIt seems that *.csv can only save one worksheet in one *.csv file. [...], we can have 6 different files because *.csv file only support one worksheet.

That is correct. you can store only one sheet into csv.

Quote[...] Unlike *.xls (excel) where we can have many worksheet in only one file.[...] Unlike *xls where it can save that 6 different files into just one file because it support multiple worksheet.

So, my suggestion that, for future FET release, FET can import and export *.xls file

That is a TODO, but i already know that i will not do that this year.

I found only unofficial xls format file descriptions (no official one from Microsoft) and i don't use Microsoft excel.
Also that file format is much more difficult than csv.

So it look like an other guy must code that if you need it quick.

Maybe you can also ask Microsoft, if they code an export function into a single cvs file? That import function will be much easier to code, because they already have an export to csv of worksheet one. They just need to add a few more lines of code to do that.
Writing an import filter (in FET) need at least several hundred lines of code.

An other solution is to use not worksheet one. use only worksheet 2 to 7. Before exporting you just copy your 6 sheets into sheet 1. You can code also an macro for this in excel so you don't need to do that always manually.
Title: Re: import and export files
Post by: kdsayang on May 08, 2008, 06:00:46 PM
Quote

I found only unofficial xls format file descriptions (no official one from Microsoft) and i don't use Microsoft excel.
Also that file format is much more difficult than csv.

Found the official Microsoft Office Binary (doc, xls, ppt) File Formats at http://www.microsoft.com/interop/docs/officebinaryformats.mspx

Quote
An other solution is to use not worksheet one. use only worksheet 2 to 7. Before exporting you just copy your 6 sheets into sheet 1. You can code also an macro for this in excel so you don't need to do that always manually.

I don't quite understand that. Can u elaborate more with more examples??

TQ
Title: Re: import and export files
Post by: Volker Dirr on May 08, 2008, 07:48:12 PM
if i understand correct, then this is your problem:

You use 6 sheets (i guess you separated the years. So sheet 1 is year 1. Sheet 2 is year 2. ...)
You problem is, that only sheet 1 is exported to csv by excel.
Sheet 2, ... are not exported.

so most easiest solution is to just copy manually the data of sheet 2 into sheet 1. Also data of sheet 3 into sheet 1, and so on.
but this way is not very clean if you want to modify later.
that is why i said: "don't use sheet 1".
so sheet one just be empty. sheet 2 should contain data of the first year and so on.

After you finished your lesson planing you just copy sheet 2 into sheet 1, sheet 3 into sheet 1 and so on.
then you can export to csv.

Of course copy manually is not pretty nice. so it is more comfortable if you copy by a macro or if you code that. You can code in excel. If i remember correct it is an "basic" language dialect. (i code ~10 years ago with excel a little bit, but i don't remember everything.) So you only need to code run through all 6 sheets and copy every line into sheet 1.
Title: Re: import and export files
Post by: kdsayang on May 20, 2008, 05:24:11 PM
Hi,

How is the latest version (5.5.5.) with import/export progress?? Can't wait to try that feature...  ;D
Title: Re: import and export files
Post by: Volker Dirr on May 21, 2008, 08:40:58 PM
i am pretty busy at the moment.

but i already modified the source. Now you can export and import also with text quotations.
so it's possible to import csv "correct" into a spread sheet by default.

you can check it if you want. i release version vd2 at my page. But source only.

Liviu already found some changes that must be done in the interface. So i need to do that before i convert to the latest version.