Using Algorithmic Taxonomy to Evaluate Lecturer
Workload
Ruhil Hayati Hashim, Jamaliah Abdul Hamid, Mohd Hassan Selamat,
Hamidah Ibrahim, Rusli Abdullah, Mohd Ghazali Mohayidin, Universiti Putra Malaysia
ABSTRACT:
Lecturer workload at universities includes three major categories: teaching,
research and services. Teaching workload is influenced by various factors such
as level of taught courses, number of student, credit and contact hour and off
campus or on campus course design. Universiti Putra
Keywords: Taxonomy, Algorithmic
taxonomy, Service workload, Scoring, university portal, Knowledge management
1. Introduction
One of the greatest assets in academic world is the intellectual property,
knowledge and experience of their academic staff. Universiti
Putra
The information amassed in the UPM KM Portal was studied to develop specific use of information in automating several key processes related to the service of the university academic staff. In this paper, the work process specifically referred to is the process of automating score count for work load. Reports of automation of processes for lecturer appraisal is discussed in another paper. The rule based algorithmic paths for the scoring processes of these functions were proposed. These paths were continuously evaluated by various university panels to check on the level of acceptability and ethical implications. The database of the UPM KM Portal is ORACLE, and the researchers used java as the programming technology.
This paper is organized as follows. Section 2.0 briefly defines taxonomy and describes its role in Knowledge Management. Section 3.0 defines lecturer workload and describes its components. A literature review of workload based on Lecturers’ Service Activities is also discussed in this section. Section 4.0 describes the methodology used to develop the taxonomy for automatic scoring of lecturer workload. In Section 5.0 the taxonomy used in the prototype to identify the overall lecturer workload in UPM KM Portal is described. This is followed by Section 6.0, where the focus is narrowed to the taxonomy for scoring of workload based on service activities. Section 7.0 describes the algorithm used in the prototype, Section 8 describes the technology deployed, and Section 9.0 contains the details of prototype system architecture, while Section 10.0 is the explanation of the functionality of the prototype. In Section 11.0, the system flow of the prototype is explained. Our study is concluded in Section 12.0. Internet sources for technology deployed are provided in the Apendix.
1.1. The Problem
The massive amount and rapid increase of information and the lack of automated co-ordination in organizing, extracting and making further use of this information is making knowledge mining more and more difficult. Metadata are disparate parts of information in the repository which require a certain structure, called taxonomy, through which they can create some logical relationship. The problem at hand is to define an appropriate taxonomy to connect various metadata in some form of logic to enable an automatic scoring for lecturer’s workload cutting across various tasks of teaching & supervision, research, consultancy & publication and services. The taxonomy model developed must also appropriately measure performance in each of these areas. In this research report, the problem is first focused on defining service workload. Then an operational taxonomy is established to reflect in totality the lecturer’s research workload. Next, an algorithm is developed to reflect the hierarchy of importance of the metadata and to accommodate for various entry points of scoring at different levels of the hierarchy. The algorithm also takes into account repeated scoring in the formula.
2. Taxonomy
Taxonomies are frequently hierarchical in structure. However taxonomy may also refer to relationship schemes other than hierarchies, such as network structures. Taxonomies are fundamental structures used in many areas of information system to explain the hierarchichal relationship of various parts of information.
Other taxonomies may include single children with multi-parents, for example, "Car" might appear with both parents "Vehicle" and "Steel Mechanisms". Taxonomy might also be a simple organization of objects into groups, or even an alphabetical list.
Mathematically, a hierarchical taxonomy is a tree structure of classifications for a given set of objects. At the top of this structure is a single classification, the root node that applies to all objects. Nodes below this root are more specific classifications that apply to subsets of the total set of classified objects. So for instance in this prototype for lecturer workload, the root is the Workload (as this applies to all lecturer). Below this is the job status of the lecturer within the university which has implications on the distribution of work load. The workload divides into teaching, research, consultancy and publication (R, C & P) and services (see Figure 1).
Taxonomy is practically represented as a tree that classifies a set of metadata at a low level into a more general metadata at higher level. Taxonomies facilitate associative thoughts and flows because they chart the hierarchical and associative relationships that exist within and between data.
Taxonomies define a world-view because they specify how categories of information are arranged in a hierarchy. The political nature of taxonomies requires careful study of user profile when selecting the hierarchical order and relationships amongst categories of information. In this case study at UPM, the taxonomies to score for workload were seriously reviewed to highlight several ethical and highly political issues which had to be resolved with staff members and university administrators.
3. Lecturer Workload
In UPM, a lecturer must have at least a Master’s degree from a recognized university in his/her field of specialization. The normal workload of lecturer shall include teaching; research, scholarly and creative activities; and service to the university in proportions of approximately 40%, 40% and 20% respectively of each lecturer's time. Workload is also defined as “all activities that take the time of the university faculty member and are related to professional duties, responsibilities and interests”.
3.1. Teaching Workload
Teaching workload include all teaching-related activities such as material preparation and actual teaching. These activities are affected by factors such as the number of students in the course, level of course, contact hours, off or on campus, and whether the course is taught through team teaching or by individual lecturers.
3.2. Research Workload
Research workload is directly related to input into conference presentations, doing peer reviews, application for external funding, administration of research project, publications of professional reports, and developing research outputs.
3.3. Service Workload
Workload also includes services offered by lecturers to the university such as administrative duties to the College, Faculty, and University and membership on committees. Service workload also includes service to various university associations, the community and to the larger society.
3.3.1. Review Of
Service Workload Models
A service workload model from Sydney University, Australia is reviewed. All
figures are intended to reflect time spent based on nominal rate of a full load
of 35 hours week, 43 weeks per year, or 1500 hours work. This means that 15 hours of work will
score 0.01 in the normalized point system. Table 1 show the points for the type
of service workload from
Service activities can be grouped into two categories: institutional service and professional service. Institutional service includes all the activities that are not related directly to teaching and research but that indirectly contribute to these missions. University administration is one of the primary areas of institutional service. In addition to the duties performed by full-time administrators and staff members, there are many administrative jobs done part-time by lecturers. Titles like associate dean, department chair, director of graduate studies, and program coordinator are held by lecturers who sacrifice part of their teaching and research responsibilities to help make the university function better, thus allowing for better quality teaching and research to take place. Committee work is another form of institutional service. Whether it is a departmental curriculum committee, a college personnel committee, a campus governance committee, or an intercampus research-review committee, the work is often difficult and time-consuming but important to the well-being of the entire intellectual community. This also includes for instance, student advising.
Professional service is usually done in support of the various academic disciplines. Lecturers who hold offices or serve on committees and boards in professional organizations, organize and chair sessions at national and international scholarly meetings, serve as editors or manuscript readers for professional journals, or participate in on-site program evaluations are contributing services to their professions rather than specifically to their home faculties or campuses. Such discipline-oriented (rather than institution-oriented) professional service usually falls to those who have distinguished themselves in research.
Various administrative duties are spelled out and accounted
for in detail in the
Overhead load is included for all staff for meeting attendance, mail, phone and 6 hours advising/registering and 20 hours exam marking per semester plus one Faculty examiners meeting.
|
Administration |
year |
hrs/week |
|
HOD |
0.40 |
14.0 |
|
Ugrad Director & Tut
Manager |
0.30 |
10.5 |
|
Ugrad Admin, Timetable, Exam admin |
0.20 |
7.0 |
|
Honours Director & Seminars |
0.20 |
7.0 |
|
PDR course Director (BIT, MInfTech) |
0.10 |
3.5 |
|
Program Marketing |
0.15 |
5.25 |
|
Chair of Departmental
Committee |
|
|
|
Resources & Space |
0.20 |
7.0 |
|
Research (Also responsible for P/g students) |
0.20 |
7.0 |
|
Education |
0.20 |
7.0 |
|
Other
Administration |
|
|
|
Overhead load |
0.10 |
3.5 |
|
External Relations & Summer School |
0.20 |
7.0 |
|
Professional Development Load: |
0.10 |
3.5 |
Table 1 - Points for the type of Service Workload from the
Services activities cover service within university and service outside university. Community service may involve organizing a professional seminar for the business community, participation in local community services (including membership on an organization board, and professional speaking to the local community). Service may also involve contributions to local, regional and national academic societies, including holding offices, organizing a conference, serving as a journal editor or referee, or founding a society or journal.
Some nodes of university service are counted either by number of years the
service is offered, or based on the of number of services
contributed from within or outside the university.
4. Methodology For
Developing The Taxonomy For Lecturer Workload
The core project methodology is based on a framework of initiating, planning, and implementing improvement. The framework is divided in five phases which are:
Initiating: This first stage of the research involved a literature survey and review of existing workload taxonomy models and lecturer workload. In this phase all the data is collected from surfing the Internet, reading journals and articles. The aim of the study is to identify types of taxonomy models. At the same time, technological tools to build taxonomies were also surveyed. The most important task in this phase was to look into the score point system used by the Registrar of UPM to calculate appraisal scores, and then to translate that score point system in the context of workload scores.
Diagnosing: This stage requires the identification of the problem, and how a Taxonomy model will offer opportunities of a solution. The purpose of the model taxonomy is to categorize all information on workload into a hierarchy, thus providing an infrastructure to organize, navigate and retrieve structured information more quickly. The immediate problem was to identify which information in the database in the KM Portal can be logically quantified, and place at what level of hierarchy (with ethical reasoning) to count the lecturer workload.
Establishing: The next step is to plan and formalize the taxonomy model for lecturer workload. A process begins with an initial model, e.g. of some university policy or metadata being presented to a few panelists to elicit their independent opinions and recommendations. These are used by the researchers as a basis to build and revise the taxonomy model. Once revised, the model is presented again to the panelists for another round of review / comment and possibly further statement revision. The process iterations at this stage aims to reach consensus for approval of an accepted version. The panelists comprised lecturers, researchers, and university administrators who are all familiar with the work tasks of lecturers. Minimum loadings of lecturer workloads are also identified. Parallel to this phase, we also did some algorithm and some coding for the prototype.
Execute: At this stage, the taxonomy model is structured according to plan. The taxonomy prototype for the automatic scoring of lecturer workload is then tested in a selected faculty.
Learning: Testing and re-testing the prototype is a learning process.
Refinements are made, and further enhancement and development work are
identified. At the conclusion of the
prototype stage, presentations are made at conferences and written in journals.
5. Case Study: Prototype Taxonomy For Automatic Scoring Of Overall Lecturer Workload In The Upm Km Portal
The Knowledge Management Portal of Universiti Putra
Figure 1 shows the taxonomy of the Lecturer Workload prototype developed for the UPM KM Portal. Three types of professional work carried out by lecturers are first identified. These are: T & S (teaching & supervision); R, C & P (research, consultancy & publication); and Services. The metaspecifications for T & S; R,C,& P; and Services are available in figures 2, 3 and 4 respectively. The prototype workload is further intensified by measurable scores for selected conditions, such as the number of credit, number of students taught, joint or single authorship publications, appointment as a member or a leader of various committees, and appointment as a member or a leader in consultancy services.
Figure
1 - Prototype model for Overall Lecturer Workload
Figure
2 - Taxonomy for workload in T & S
Figure
3 - Taxonomy to count workload in R,C & P
6. The
Taxonomy For Computing Workload Scores Based On
‘Service Activities’ In The Prototype Model
As described earlier, the root node is
labeled by Workload. (from Figure 1). It denotes the most general metadata
class. Figure 4 shows an example of taxonomy of workload scoring based on
Services, where ‘Service Within University’ and ‘
A level can be assigned to each node in the
taxonomy. The level of the root is zero, and the level of any other node is one
plus another to get the sum score for its parent.
Formula to count
Services = Within University +
The count formula
for Within University node is:
Within University =
Number of service rendered to the university in a year, where some services can
be mutually exclusive.
Mutually exclusive
Yearly defined services = Σ1(Dean/Director or Principal or
Deputy Dean/Director or HOD) where only one of these positions can be taken in
a year. The value of a role can also be 0.
|
Yearly Services |
Score |
|
Dean/Director |
12.0 |
|
Principal |
10.0 |
|
Deputy Dean/Director |
9.0 |
|
HOD |
8.0 |
|
Coordinator |
4.0 |
|
Chief/Member Board of Periodical Editor or web site or
equivalent |
3.0/2.0 |
Table 2 - Score for
every node in Yearly- Services
Non mutually exclusive service are services
which can be simultaneously rendered to the university, and they include
chairmanship, secretariat, and committee membership (cscm), seminar organizers (so)
and so on. Sport Participation is denoted as sp, chair or member of Evaluator Board denoted by eb; chair or committee of thesis
examination by te, professional
services by ps, program assessors as pa, academic advisor to number of
students as aa, and main or committee
supervisory member as mcs. Therefore,
non mutual exclusive services= Σ2[({cscm1+cscm2 …+cscmn}
* (4.0 or 3.0 or 2.0))+({so1+so2+…son} * (4.0 or 3.0 or 2.0))+({
sp1+sp2+…spn} * 3.0)+ ({eb1+eb2+…ebn} * (3.0
or 2.0))+ ({te1+te2+…ten} * (2.0 or 3.0))+ ({ps1+ps2+…psn}
* 2.0) )+ ({pa1+pa2+…pan} * 4.0) )+ ({aa1+aa2+…aan}
* 2.0) )+ ({mcs1+mcs2+…mcsn} * (3.0 or 2.0))] where value of
cscm, so, sp, eb, cp, ps, pa, aa and mcs can also be 0.
cscml=Number of role
as Chairman/Secretariat/ Committee Member;
so = Number of role
as Chairman/Secretariat/Committee Member seminar organizer;
sp = Number of Sport Participation;
eb = Number of role as Chairman/Member
– Evaluator Board of Academy Study;
te = Number of role as Chairman/Member
– Thesis Examiner;
ps = Number of Professional Service;
pa = Number of role as Program Assessor;
aa = Number of role as Academic Advisor;
mcs = Number of role as Main/Committee Supervisor
To give an illustration of the formula for
Services (Within University), a workload for two lecturer scores is computed in
Table 3.
|
|
Lecturer A |
Score |
Lecturer B |
Score |
|
Yearly - Services |
Director |
12.0 |
Deputy Dean |
9.0 |
|
|
Chief - Board of Periodical Editor or web site or
equivalent |
3.0 |
Member -Board of Periodical Editor or web site or
equivalent |
2.0 |
|
Number - Services |
Chairman - Member seminar organizer * 2 |
4.0 * 2 = 8.0 |
Sport Participation * 3 |
3.0 * 3 = 9.0 |
|
|
Academic Advisor * 3 |
2.0 * 3 = 6.0 |
Academic Advisor * 4 |
2.0 * 4 = 8.0 |
|
|
Total Score Lecturer A |
29.0 |
Total Score Lecturer B |
28.0 |
Table 3 - Calculation of
Services (Within University) workload
Figure
4 - Taxonomy for workload in Services
7. Algorithm For Workload
Prototype Model
For the prototype, we use the following scheme which summarizes this algorithm:
1. load the data from the database
2. create a new tree
3. create root and parent for the tree
parent_id = 0;
parent_id = root;
current = n;
4. create child
int returnValue;
if root == null
root = n;
n.setLevel (0) - n
will set the level
else
n.setParent (n) - n
will set the parent
TreeNode firstChild = n.getFirstChild();
TreeNode lastChild = n.getLastChild();
n.setLevel (n.getLevel()
+ 1);
setNode (lastChild);
if (lastChild ==
null )
current.setLastChild(n);
else
lastChild.setNextNode(n);
return returnValue;
5. child (returnValue) will add to the parent
6. to count scoring for every child is
child(n) = score x
quantity;
7. to count scoring for every parent
Sum += child(n);
8. Display every item for root; parent; child; with their scoring, quantity and total of scores.
8. Prototype Technology
The software technology used in this prototype is: JavaServer
Pages, Apache Tomcat, Oracle and Macromedia Dreamweaver.
8.1. JavaServer
Pages (JSP)
As part of the Java technology family, JSP technology enables rapid development of Web-based applications that are platform independent. JSP technology separates the user interface from content generation, enabling designers to change the overall page layout without altering the underlying dynamic content.
8.2. Apache Tomcat
Apache Tomcat is the servlet container that is used in the official Reference Implementation for the Java Servlet and JavaServer Pages technologies.
8.3. Oracle
Oracle Database is designed for enterprise grid
computing. It also enables numerous
quality and performance enhancements of data management. Oracle Database 10g
significantly reduces the costs of managing the IT environment, with a
simplified installation, greatly reduced configuration and management
requirements, and automatic performance diagnosis and SQL tuning. It is
relatively easy to deploy other automated management capabilities onto ORACLE
to help improve DBA and developer productivity and efficiency.
8.4. Using JDBC ODBC
The JDBC API is the industry standard for database-independent connectivity between the Java programming language and a wide range of databases. The JDBC API provides a call-level API for SQL-based database access. JDBC technology allows the use of Java programming language to use “Write Once, Run Anywhere" capabilities for applications that require access to enterprise data.
8.5. Macromedia Dreamweaver MX
Dreamweaver MX 2004 is the professional choice for building web sites and applications. It provides a powerful combination of visual layout tools, application development features, and code editing support. Dreamweaver enables web designers and developers to easily create and manage any website.
9. Prototype System
Architecture
The overall system architecture, as illustrated in figure 5 below, consists of end-users accessing the system through any browser or WWW client such as Netscape or Internet Explorer. The prototype system deploys a UNIX server and defines databases using the commercially available database management system ORACLE.
Figure 5 - Standard retrieval on
Internet
The prototype system is implemented using client-server architecture on local area network (LAN). The server is an Apache running Window XP. The server machine also has ORACLE database management system with databases installed. This database defines tables and views. The design of the database is beyond the scope of this paper. Method is constructed so that an authorized user can edit and enter records into this database.
10. Functionality Of
The Prototype
The prototype system locates and retrieves available on-line standards specifications and documentation. The demonstration system supports one type of result which is : Retrieval from a database, guided by the taxonomy.
The search based on the taxonomy starts with the high level context of a user's information processing profile in the Curriculum Vitae (CV) Module. For example, to compute for service workload, the system will count the workload from the CV module in the database.
Information processing profiles are defined in this paper as a hierarchical tree-structured (taxonomy) collection of services needed to support applications which address an exact task area e.g. task area of teaching workload. The taxonomy lets the user simplify the task areas from a broader down to a narrower area which aids in determining what further system specific standards are required. The higher level of taxonomy development is searched by root and goes to workload which is T & S; R, C & P; and Services. Thus, the research workload result of every node will be presented. See the figure 6:
Figure
6 - The output from the prototype
An enhanced feature for the taxonomy will terminate with a list of scores for selected node. Once the list of workload item is identified, a listing of the node score, quantity, and total scores are displayed.
11. System
Flow
Figure 7 below illustrates the system implementation flow. The navigation through the taxonomy allows the user to select from all possible subject areas of workload, to arrive at and obtain summary information about the specific standards needed for the application area. As the list of retrieved standards is presented, the user can specify what information about each standard is of interest for workload.
Figure 7- The system implementation flow
Each successive web page is constructed dynamically based upon the users input into the database in KM Portal. The parameters are processed by a Java Servlet coding written in Java Server Pages (JSP).
The Java Servlet coding formulates SQL database queries which are sent to the ORACLE database. The retrieved results in the form of "raw" data are formatted in HTML and sent back to the user.
This system allows the users to access the database information by using only normal Web browsers. But additional software or java plug-ins at the users site are required.
12. Conclusion
This paper develops a prototype that identifies and characterizes metadata
for lecturer workload. This prototype system facilitates the retrieval of
metadata or information from database for service workload. The workload
taxonomy prototype supports decision making for administrative purposes. The
system represents a useful and convenient information retrieval and weighted
scoring without the need for special software at the users’ site. This
prototype demonstrates how structured data can be taxonomized
to enable unsupervised or automated triggers for weighted searches.
Lecturer workload is an important consideration since it can affect lectures’ confidence and commitment to quality work outcomes. If a faculty member perceives workload policies to be unfair, then that lecturer may not perform to the highest potential; if confidence is severely affected, the lecturer may even decide to relocate to another institution. Fair and equitable work load distributions with corresponding rewards are necessary if institutions want to keep their best lecturers.
Lecturer performance can be evaluated by quality and by quantity. Quantifying lecture workload is a good preliminary effort to achieve fair and equitable evaluations. This paper presents a first step towards creating a comprehensive, quantitative workload policy. A good workload policy should also coordinate with and reinforce strategic goals and promotion and tenure standards. The prototype is a first step in this direction. It is not intended to be a final, model document, but rather an invitation to a system to be integrated within the knowledge management portal of the university. Future research may revise this taxonomy or apply it in addressing a variety of workload issues as described in the previous section.
13. References
Chuang, S.L., Chien, L.F., 2003, “Enriching Web taxonomies through subject categorization of query terms from search engine logs”, Decision Support System, 35, 113-127.
Comm, C.L., Mathaise, D.F.X., 2003, “A case study of the implications of faculty workload and compensation for improving academic quality”, The International Journal of Educational Management, 17(5), 200-210.
Ellingson,
Henninger, E.A., 1998, “Perceptions of the impact of the new AACSB standards on faculty qualifications”, Journal of Organizational Change Management, 11(5), 407-424.
Higgins, J. C., 1989, “Performance measurement in universities”, European Journal of Operational Research, 38, 58-368.
Holsapple, C. W., Joshi, K.D., 2001, “Organizational knowledge resources”, Decision Support System, 31, 39-54.
Hughes, C., 1999, “Faculty publishing productivity: the emerging role of network connectivity”, Campus-Wide Information Systems, 16(1), 30-38.
Lackritz, J.R., 2004, “Exploring burnout among university faculty: incidence, performance, and demographic issues”, Teaching and Teacher Education, 20, 713-729.
Mancing, H., 1991, “Teaching, Research, Service: The Concept of Faculty Workload”, ADFL Bulletin, 22(3), 44-50.
Mancing, H., 1994, “A Theory of Faculty Workload”, ADFL Bulletin, 25(3), 31-37.
Olive, A., Teniente, E., 2002, Derived types and taxonomic constraints in conceptual modeling, Information Systems, 27, 391-409.
Simon, J.C., Soliman, K.S., 2003, “An alternative method to measure MIS faculty teaching performance”, The International Journal of Educational Management, 17(5), 195-199.
Sridhar, S., 1998, “Decision support using the Intranet”, Decision Support System, 23, 19-28.
14. Appendix
http://cobe.boisestate.edu/govern/workload.htm
http://ironbark.bendigo.latrobe.edu.au/staff/tim/Workloads/WorkloadSystem.doc
http://java.sun.com/products/jsp/
http://tomcat.apache.org/index.html
http://www.aau.org/studyprogram/pdfiles/ruthd.pdf
http://www.canberra.edu.au/hr_old/policies/summary_draft.html
http://www.csse.monash.edu.au/~hws/load.html#Formula
http://www.kimep.kz/section1.htm
http://www.macromedia.com/software/dreamweaver/
http://www.mla.org/adfl/bulletin/v22n3/223044.htm
http://www.oracle.com/index.html
Contact the Authors:
Ruhil Hayati Hashim, Research Officer in Center for Academic Development,
Universiti Putra Malaysia1
Jamaliah Abdul Hamid, Deputy
Director of Knowledge Management Division, Center for Academic Development, Universiti Putra Malaysia1
Mohd Hassan Selamat, Assoc. Prof. in the Faculty of Computer Science, Universiti Putra Malaysia2
Hamidah Ibrahim, Assoc.
Prof. in the Faculty of Computer Science, Universiti Putra Malaysia2
Rusli Abdullah, Lecturer in the Faculty of
Computer Science, Universiti Putra
Malaysia2
Mohd Ghazali Mohayidin, Director, Center for Academic Development, Universiti Putra Malaysia1
1Center for Academic Development, 4th Floor,
2Faculty of Science Computer & Information Technology, Universiti Putra