Engineering design is an ill-structured problem solving and open-ended task (Dym et al., 2005) because design problems have ill-defined goals, states and solution steps. Engineering graduates are expected to design solutions to open-ended real world problems. Due to the complex nature of engineering design, the teaching and learning of this skill is reported to be difficult (Dym et al., 2005). Conceptual design is an important and critical step in design (Pahl et al., 2013). Conceptual design is described as a process in which the functional requirements of the design problem are transformed into descriptions of solution concepts (Chakrabarti & Bligh, 2001). Although all the processes in design are vital for the end result, a strong case can be made for selecting the conceptual design as most critical to the final design (Chakrabarti & Bligh, 2001). The conceptual phase of design thus becomes very significant, as designers tend to develop numerous early ideas and solutions in this phase.

Software design has several common activities with other design domains (Cross et al., 1996). However, the dynamic and intangible nature of software poses unique challenges in software conceptual design (SCD), specifically, components are logical and intangible, and behaviours of such intangible components need to be simulated along with simulation of end-users interactions (Petre et al., 2010). Experts create integrated solutions that fulfil the requirements. Novices find designing software solutions for open-ended problems daunting. There have been previous studies of novice difficulties (Eckerdal et al., 2006), however the underlying mechanism that causes these difficulties has yet to be unearthed. Moreover the ways to alleviate them in the context of SCD have not been reported. Current teachinglearning methods do not explicitly train students to overcome these difficulties (Armarego, 2009). There is a need to understand novices’ design processes and explicitly train computer-engineering students in SCD. This is the motivation of this thesis; firstly, to develop an understanding of novices’ design processes in SCD and secondly use this understanding to design supports for novices’ to create integrated SCD.

We used the function-behaviour-structure (FBS) design framework (Gero & Kannengeiser, 2014) as a lens to analyse novice processes as well as support the 6 creation of SCD. We followed a design based research methodology (Barab, 2014). We started with understanding novices’ design processes, the design strategies and cognitive processes. To identify these, we used protocol analysis (Gero et al., 2011) with novice computer engineering students (Study 1), to collect data, as they create software conceptual design for open-ended problems. We found that novices are fixated to a single view of the software solution and unable to utilize multiple formal representations of UML to model SCD. Additionally the solutions that novices create lack integration.

Evaluating a software design is an important practice required of software engineering students. When students graduate and enter the software industry, they usually work on large existing systems, and spend their first several months resolving bugs and writing additional features based on new requirements. This requires students to comprehend an already existing design, incorporate the required feature in the design, and evaluate if the design satisfies the intended goals. The design of a software system is often specified as a set of Unified Modelling Language (UML) diagrams, which describe different views of the system, such as the structural view (e.g. class diagrams), and the  behavioural view (e.g. sequence diagrams). However, students face difficulties in understanding how the overall system specifications actually work based on these views. Moreover, current software design courses do not place sufficient emphasis on teaching students how to evaluate designs.

The broad research goal of this thesis is to “Design and develop a technology-enhanced learning environment (TELE) which enables students to evaluate a software design against the given requirements”. Evaluating a given design can be viewed from different perspectives. These include checking for its syntactic quality (whether the UML diagrams are modelled according to the syntax of the language), semantic quality (how well the design maps to the given requirements), or pragmatic quality (how well a given design can be interpreted by different stakeholders). In this thesis, we have focussed on enabling students to evaluate a design by checking for its semantic quality. This requires students to think deeply about the design, and understand the relationship between different diagrams in the design.

In order to answer this research goal, we began by analysing existing literature to identify student difficulties in the design process, as well as practices and strategies experts use to evaluate a given software design. We then conducted two studies with students to understand how they evaluate a given software design, and the difficulties they face. The key insight which we gained from the literature review and novice studies is that effective evaluation of design diagrams depends on the quality of mental models that students create based on the requirements and the design. Experts create a rich mental model of the system and simulate various scenarios of system behaviour in the design. Experts’ mental models contain information regarding the control and data flow on simulation of such scenarios. However, novices were unable to simulate such scenarios, and their models focussed on superficial aspects of the design.

These insights form the basis of the VeriSIM (Verifying designs by SIMulating scenarios) pedagogy. VeriSIM trains students to identify and model scenarios in the design. The VeriSIM pedagogy comprises two strategies - the design tracing and the scenario branching strategy. In the design tracing strategy, students construct a model of the scenario, which is similar to a state diagram. They trace the control and data flow of a scenario while constructing the state diagram. In the scenario branching strategy, students identify different scenarios from the requirements by constructing a scenario tree. Traversing the scenario tree enables them to identify scenarios which do not satisfy the requirements.

The VeriSIM pedagogy has been operationalized into a technology-enhanced learning environment having two modules. In Module 1, students go through design tracing activities in the VeriSIM learning environment. In Module 2, students go through the scenario branching activity using a mapping tool, which is facilitated by an instructor. We believe that both the broad exploration of the design by identifying scenarios, and a deep understanding of each scenario by simulating the data and event flow for that scenario, can lead to effective evaluation of a given software design.

Historical thinking is the set of thinking skills required for learning history or doing history.  Experts have argued that learning historical thinking is a slow process due to its counter-intuitiveness and requiring explicit engagement.  Further, novice learners have difficulty answering questions requiring historical thinking as they are overwhelmed by verbose and complex texts.  The first step for any reading - writing task is the organisation of data for sense-making.  Thus, for novice learners the study of history is conventionally limited to curated pieces of text, the memorisation of facts, figures, events, dates, concepts, timelines and answering questions at the memorise level rather than questions requiring higher order thinking.  This dissertation contextualises itself in a teacher-as-facilitator environment.  It addresses the problem of 'being overwhelmed' by providing a novel technique to help novice history learners organise text to create meaningful visual relationships to aid sense-making of the linear text thus reducing the information overload that hinders the learning of historical thinking.  Subsequently, the learners are tested on their ability to answer questions requiring historical thinking.

Supported by the theories of external representation, external and distributed cognition, and cognitive flexibility theory, this dissertation shows that if novice learners of history are provided with scaffolding to support graphical organisation and reorganisation of textual data then they would be able to better answer questions of historical importance.  To achieve this, a set of notations to support concepts in History were designed along with a technology tool called History-Maker, to leverage digital affordance.  Learning tasks that required four historical thinking skills, namely corroboration, contextualisation, claims and evidence, were provided to the learners along with appropriate rubrics for scaffolding and feedback.

A two cycle three stage Design Based Research (DBR) methodology was adopted for this dissertation.  Within the overarching DBR methodology, Design Thinking methodology was applied to the 'problem analysis and design of solution' within the first cycle as the solution had to be designed in a "designerly way".  In total nine studies were conducted.  The first DBR cycle, comprising of studies one to seven, identified the problem from literature. The problem was replicated through a study contextualised to history and historical thinking. Paper-based solution was then designed and evaluated.  The second DBR cycle, comprising of studies eight and nine, identified the limitations of the paper based design, a digital tool called History-Maker was created, and the technology tool based solution was evaluated.  Study One identified the difficulties that novice learners of history face when asked to solve a question requiring historical thinking.  Visual notations specifically adapted for history were designed and validated by collective participation of multiple stakeholders.  Studies two to five adapted, designed and tested the abstract notations for historical thinking.  Studies six \& seven evaluated the paper based solution and its effect on learning historical thinking using both quasi-experimental and experimental methods. A digital tool called History-Maker was designed, developed and evaluated to provide the benefits of digital affordances. Study eight was an evaluation of the History-Maker tool by history professors. Study nine evaluated the history-maker tool in enabling the learners to organise data while learning historical thinking.  The effectiveness of History-Maker, its usability, usefulness, and motivation of the participants, were evaluated as well.

In an ABAB design study (study seven) (N=73) with first year under-graduate history learners, it was found that the novice history learners engaged in a learning activity significantly improved in their historical thinking ('corroboration', 'contextualisation', 'claims' and 'evidence') scores while engaged in organising and reorganising the complex and verbose historical data using history-specific visual notations on paper.  Further, a single group pre-post test non-generalisable study (study nine) (N=25) with another group of first year under-graduate history learners using the History-Maker digital tool showed that scores on tasks related to 'contextualisation', 'claims' and 'evidence' had a significant improvement. Thus, the dissertation provides substantial evidence to show that history specific abstract notations (historical thinking notations) to visually organise and reorganise complex and verbose textual historical data reduces information overload and enables sense-making thus significantly improving learning historical thinking.

Bioscience practitioners regularly evaluate the effect of a phenomenon across biological levels, understand the patterns of inheritance, study structure, function, and growth of living organisms and others (Hoskinson et al. 2013). To do this, they have to integrate various domain-related concepts, and science inquiry practices. To perform such scientific exercises, an undergraduate learner is required to integrate concepts and practices by performing a complex cognitive process during problem-solving, which is required in industry/higher studies (van Merriënboer & Dolmans, 2015). Besides this, he/she is also expected to transfer and apply this learning to a novel scenario. Undergraduates learn the concepts and practices in silos as part of theory classes, practical labs, and tutorial sessions. Hence it is difficult for them to connect the three, and there are very few instances in the existing curriculum where they are explicitly asked to do so. So there is a need for a teaching-learning solution for problem-solving that facilitates the integrations of concepts and practices learned as part of the curriculum, which is also aligned to the existing curriculum.

We have used design-based research (DBR) as our overall research framework to develop this solution. DBR is a research methodology that aims at the development of educational interventions or learning environments through iterative cycle of analysis and exploration, design and development, and evaluation and reflection. The research activities are based on collaboration among researchers and practitioners in real-world settings which leads to contextually-sensitive design principles and theories. We have executed two research cycles of DBR, where each of the cycles involved the research activities of problem analysis, solution design, evaluation, and reflections. Our first research goal was to understand learner‘s difficulties during problem-solving. To identify these, we performed an exploratory study with bioscience instructors (Study 1) as they solved the open-ended problem in genetics and identified the learner‘s difficulties. We also identified the concepts required to solve the given problem and curriculum alignment. We found that the concepts and science inquiry practices needed to solve such problems are spread across the curriculum. The concept of genetics a learner learns in their first year, the concept of biostatistics in the second year and science inquiry practices as part of practical labs. The instructor study and literature review together helped us identify the concepts and scaffolds needed to facilitate the integration of concepts and science inquiry practices during problem-solving.

We designed, developed and evaluated GI (Geneticus Investigatio), a technology enhanced learning environment (TELE) to scaffold learners‘ problem-solving. The key pedagogical features and learning activities in GI include set of inquiry-driven reflective learning experiences. A learner performs authentic inquiry-based learning activities with reflection at various stages during problem-solving. These features have been included based on the pedagogical theories of inquiry and authentic learning, question prompts and types of scaffolds. Broadly, the pedagogy is designed based on the intertwining of cognitive and metacognitive tasks in such a manner as to scaffold integration during problem-solving. GI has affordances such as access to domain concepts, set of evaluative questions with feedback. The entire interactive process of learning with GI allows repetition of a task until mastery is reached. The GI-TELE was evaluated in a lab study (Study 2) wherein the learning gain was measured using the pre-post analysis, and thematic analysis was done to study how learners used the features in the TELE to solve the given problem. Based on this evaluation, we revised our design and then evaluated the revised TELE in a field study (Study 3). Here again, we evaluated GI through a quasi-experimental study and thematic analysis to study how the revised features supported learners during problem-solving. It was followed by Study 4, wherein the GI module was tested for another topic, and the effect of multiple interactions with GI module on learner‘s learning was also evaluated in study 5.

Five research studies (N=304) using a mixed-method approach were carried out as part of this research. The participants mostly were learners from the second and third year of bioscience courses from colleges affiliated to the University of Mumbai, a large public urban university in India. Results showed that learners who interacted with GI develop an integrated understanding of domain concepts and science inquiry practices during problem-solving compared to a control group. Interaction with the GI module is especially beneficial for the low scorers. We also identified features of GI that enable learners to effectively use GI for achieving the objective of integrating concepts and practices. Through the two DBR cycles, we designed, developed and evaluated the following: (1) a pedagogy known as Geneticus Investigatio for facilitating integration during problem-solving and (2) a web-based self-learning environment known as GI-environment, as an operationalization of the GI-pedagogy. The thesis contributes by (i) designing a GI-pedagogy for scaffolding integration during problem-solving (ii) GI-TELE, which has been evaluated through multiple studies. Three modules of GI has been created in the domain of basic genetics.

Software design problems are ill-structured, in which the problem space and solution space are not well-defined. In problem space, the requirements are not well defined, and a designer has to formulate incomplete requirements into specific data models, main software functions, and sub-functions. In solution space, there may be multiple solution paths, alternative design options, and the criteria to evaluate and select an optimal solution may not be clearly stated. The issues in software design are that the quality of the design is heavily dependent on the expertise and experience of the designer. Novices tend to think at the programming level and reduce early to the solution design, which affects the quality of the design in many ways.

Research studies have shown that systematically expanding and reducing the problem-solution spaces improves the quality of the solution design. In this thesis, we refer to the ability to expand the problem-solution space and eventually reduce towards a solution, as expand-reduce (ER) skills. In problem space, the ER skill is the ability to explore the problem as a whole and reduce the problem into sub-problems based on the goal to be achieved. In solution space, the ER skill is the ability to generate alternative solutions and reduce to one solution by evaluating and selecting based on selection criteria.

The existing research studies have established the importance of ER skills in solving ill-structured problems, but not much research is done in the direction of teaching-learning of ER skills to novices. To address this research gap, the broad research objective of this thesis is: ―teaching-learning of ER skills to novices using technology-enhanced learning environment.‖ Our research is scoped to teaching-learning of ER skills to undergraduate computer engineering students in the context of solving software design problems using appropriate data structures and algorithms.

We have used design-based research (DBR) methodology to design the technology-enhanced learning environment, named Fathom, for teaching-learning of ER skills. We have completed three DBR cycles to design-evaluate-redesign our intervention based on the feedback from learners and practitioners. Four research studies were carried out, which included two exploratory studies and two pretest-posttest experimental studies (Ntotal=200). Both quantitative and qualitative data were collected and analyzed. Quantitative data was used to measure the learning of ER skills by comparing the learner‘s performance in pretest, intervention, and posttest. Qualitative data in the form of log data, screen capture, and focus group interviews were used to analyze the behaviors exhibited by learners while interacting with the learning environment and student perceptions about learning ER skills.

The results showed that the Fathom was effective in learning ER skills for novices. The major contributions of this thesis are: providing insights about the cognitive biases of novice towards applying ER skills, identification of effective cognitive and metacognitive scaffolds in technology-enhanced learning environment, identification of ER cognitive tools and, the development of learning environment for learning ER skills, in the context of software design.

Engineers routinely make estimates of physical quantities such as power before they begin designing or making (Dym et al., 2005). In order to estimate a quantity, say power, a solver needs to make a simplified model, i.e., an equation relating power to parameters that significantly impact its value in the given real-world system (Mahajan, 2014). This is challenging for students because they must apply conceptual knowledge to a real-world system, identify the parameters that will dominate power requirements, make assumptions and make judgments regarding numerical values (Linder, 1999). Thus estimation is an ill-structured problem, very different from the well-structured problems which remain the emphasis of engineering curricula (Jonassen et al., 2006). Research has found a marked difference between the estimation performance of expert engineers and graduating engineering students (Linder, 1999). However, there is a dearth of research exploring the processes underlying the good estimation performance of experts. Thus, there is a need to understand these processes and explicitly train engineering students in estimation problem solving. While some researchers (Linder, 1999; Mahajan,  2014; Shakerin, 2006) have offered guidelines for learning estimation, these guidelines have not been empirically validated for their effectiveness for learning estimation. Thus the motivation of this thesis is twofold; firstly, to develop an understanding of good estimation processes and identify the cognitive mechanisms underlying good estimation and secondly, use this understanding to design supports for novices to do engineering estimation.

We followed a design-based research methodology (Reeves, 2006) with two iterations. Our first research goal was to understand estimation problem solving, i.e., what it means to do good estimation and what are the cognitive mechanisms underlying good estimation. To identify these, we performed a cognitive ethnography (R. Williams, 2006) of expert engineers (Study 1) as they solve estimation problems and identified their estimation process. We also identified the cognitive mechanisms which facilitated experts in doing estimation. We found that modelbuilding via mental simulation is the key estimation process of experts. Next, we performed a cognitive ethnography of novice undergraduates (Study 2), who solved estimation problems without any support in order to understand their estimation process, to identify differences from
the expert process and their challenges in doing estimation. We found that novices follow a process of model-searching rather than model-building, focus on equation manipulation rather than mental simulation and have difficulty with model contextualization. The expert study also showed that identifying the causal relationships of the parameter to be estimated with other parameters is an important aspect of estimation. Hence we performed a lab study (Study 3) with novice undergraduates who solved estimation problems with a simple causal mapping tool. Interaction (Jordan & Henderson, 1995) and thematic (Braun & Clarke, 1996) analyses enabled us to identify where and what scaffolds are needed to support novices estimation problem solving. The expert and novice studies together helped us identify the scaffolds needed to support novice estimation problem solving.

In order to support novice estimation problem solving, we used the insights from studies 1, 2 and 3 to design Modelling-based Estimation Learning Environment (MEttLE), an open-ended technology-enhanced learning environment (TELE). Broadly, the pedagogy is designed based on the intertwining of cognitive and metacognitive tasks in such a manner as to support learning of estimation while solving an estimation problem. Specifically, MEttLE is based on triggering learners to build models for solving estimation problems, by providing them explicit modelbuilding sub-goals and affordances such as simulations, a causal mapping tool and an equation builder. In addition, learners are provided guidance regarding expert estimation practices to make comparisons and judgments, choose values and evaluate their estimates. Finally, there are intermittent metacognitive prompts and scaffolds for evaluation, planning, monitoring and reflection. 

The design was evaluated in a lab study (Study 4) wherein we applied interaction analysis to study how novices used the features in the TELE to solve an estimation problem. Based on this evaluation, we revised our design and then evaluated the revised TELE in a field study (Study 5). Here again we applied interaction analysis to study how the revised features supported novices in solving the estimation problem. Thus, in constantly refining our design to better support novices to solve estimation problems, we refined our understanding of what it means to do good estimation, how experts are able to do it well and how we can support novices in estimation problem solving.

The major contributions of this thesis include a detailed characterization of the expert and novice estimation process and its underlying cognitive mechanisms; a set of scaffolds necessary in any learning environment that supports novice estimation problem solving and a model for solving estimation problems that leads to good estimates.

Engineering happens in laboratories hence the experiments students perform in the instructional laboratories should be carefully designed so that they become able experimenters, gain the desired knowledge and develop the necessary skills and attitudes. The experiment designs play an important role in the achievement of the laboratory learning outcomes. The tasks the students can perform in the virtual laboratories can lead to achievement of learning objectives at higher cognitive levels and certain skills and cognitive abilities such as manipulative, investigative, problem solving. This is possible due to the advanced features of the virtual laboratories. The engineering instructors should design student centered effective experiments based on scientifically proven instructional strategies and assign tasks exploiting these advanced features of the virtual laboratories. The instructors perceive that they will be able to design effective virtual laboratory experiments if comprehensive and specific guidelines are available. This problem led to the main objective of the research that is design and development of guidelines for virtual laboratory experiment design. The objective was achieved by following the three step process of Need and Problem Analysis, Solution Design by S-D-I-V-E Methodology and Evaluation.

The survey studies carried out with engineering instructors gave an insight into the aspects of the experiment design process for which the instructors need guidelines. These are: Selection of Broad Goal, Formulation of learning objectives, Designing experiments at different difficulty levels for the Expository Instructional Strategy, incorporating active learning methods within the Expository Instructional Strategy, designing experiments with Discovery, Well-Structured Problem Solving and Problem-Based Instructional Strategies, designing authentic assessment, using features of virtual laboratories to achieve the target learning objectives. The quality of each guideline was assessed based on eight criteria. The S-D-I-V-E Methodology was used for arriving at the solution.

The summative evaluation of the solution was carried out for the metrics of: Usability as perceived by engineering instructors, Usefulness as perceived by engineering instructors, Effectiveness with respect to the quality of experiment designs and Effectiveness with respect to impact on students’ laboratory learning outcomes. The results of the studies indicate that the engineering instructors perceive that the virtual laboratory experiment design guidelines as usable and useful. There is an improvement in the quality of the experiment designs after using the guidelines. There is also an improvement in the laboratory learning outcomes of UG engineering students.

After introduction in the Chapter 1 the literature review is discussed in the second chapter. In Chapter 3 the methodology adopted for the work that is the (S-D-I-V-E) Scoping-Development-Internal Review-Validation-External Use is presented and the various questions this thesis addresses are stated. Chapter 4 deliberates the various studies as part of the need and problem analysis. In Chapter 5 the details of the design and development of the solution design are provided. Chapter 6 presents the
summative evaluation of the solution design carried out by means of five studies. Chapter 7 concludes the various sections and provides discussion regarding the generalizability of the work and various limitations. In Chapter 8 the contributions of the thesis are listed down and final reflection is discussed. The various Appendices
added at the end of the thesis give details of the online SDVIcE tool, the instruments used for the studies, the rubric used to assess the quality of the experiment designs, bank of tasks and assessment questions for the BAE course, sample experiment designs and sample answer worksheets submitted by students.

When students encounter new knowledge, it is often fragmented and not well connected with their existing knowledge. Knowledge fragmentation is often larger for a learner who is new to a topic. Supporting knowledge integration (KI) is crucial to overcoming learners’ knowledge fragmentation. Moreover, better KI ensures deeper conceptual understanding of a science topic. This thesis aims to explicitly target the improvement of learners’ cognitive processes of KI. 

KI has been defined as, "the process by which learners sort out connections between new and existing ideas to reach more normative and coherent understanding of science.” It is recommended that instruction should support at least following cognitive processes: (i) Eliciting prior knowledge that may be related to the new knowledge; (ii) Focusing on the new knowledge; (iii) Distinguishing ideas - identifying conflicts, inconsistencies, and gaps. Prior research typically aim at devising instructional supports for KI for specific topics. However, this thesis targets the improvement of learners’ cognitive processes of KI, which once improved, may be applied in different topics, even if it has been acquired through another topic. Our broad research objective is: “Designing and evaluating a technology-enhanced environment (TEL) environment to improve learners’ cognitive processes associated with knowledge integration.”

Our solution is based on using exploratory question-posing (EQP) as a cognitive tool for fostering cognitive processes of KI. EQP is a kind of question-posing wherein learners pose questions with an aim to explore more knowledge around a given set of knowledge. We have empirically found that to do EQP a learner needs to link knowledge pieces from the given new knowledge and her/his prior knowledge. This means that linking leads to EQP. However, EQP may further lead to more linking which can be considered as a positive feedback loop.

The primary research question that we answer in this thesis is: “How to employ EQP in a TEL environment to improve students cognitive processes associated with KI in a Data
Structures course?” Our field studies have been administered in a number of topics in the domain of Data Structures. The target population of this research are first and second year engineering undergraduates. The artifacts produced are applicable to the Data Structures and similar domains.

We have used design based research (DBR) as our overall research framework. DBR is a research methodology that aims at the development of educational interventions and/or learning environments through iterative analysis, design, development, implementation, and evaluation. The research activities are based on collaboration among researchers and practitioners in realworld settings, and they lead to contextually-sensitive design principles and theories.

We have executed two research cycles of DBR where each of the cycles involved the research activities of problem analysis, solution design, evaluations and reflections. By the end of the two DBR cycles we designed, developed and evaluated the following: (1) An EQP-based pedagogy known as “Inquiry-based Knowledge Integration Training (IKnowIT) - pedagogy” and (2) A web-based self-learning environment known as “IKnowIT-environment” as an operationalization of the IKnowIT-pedagogy. In addition to the IKnowIT-pedagogy and the IKnowIT-environment the thesis contributes by: (1) Identifying the frequently-employed EQP-strategies which explain how learners integrate different knowledge pieces to arrive at any exploratory question in the domain of data structure and (2) Extracting local learning theories that explain how learners engagement with the features of the IKnowIT-pedagogy, including question-posing and EQP-strategies, lead to the improvement of cognitive processes of KI in them. The results show that the designed IKnowIT-pedagogy successfully fosters learners’ cognitive processes of KI using EQP.

First-year undergraduate students in engineering often face difficulties in learning and solving the engineering drawing (ED) problems that require visualization of three-dimensional objects. Conventional methods of teaching ED require students to learn the course for a semester-long duration, and it involves them to do the practice of sketching and drawing for longer durations. Certainly, these teaching methods assist the learning of a subject but do not guarantee the elimination of the learning difficulties entirely (Akasah & Alias, 2010; Kosse, 2005).

In modern teaching methods, instructors make use of software tools such as computer-aided design (CAD), multimedia tutors, and web-based instructions as a supplementary visual aid for learning of ED concepts (Branoff & Mapson, 2009; Cincou, 2013; Jerz, 2002). These methods found to be useful in the teaching of the engineering drawing course and involve various additional activities like content presentations with voice-over, demonstrations of software, and videos of sketching. Despite the fact that these methods are useful in improving the learning of ED concepts and skills; certain difficulties remain (Kuang & Thomas, 2004). One of the potential reasons is students’ deficiency in visualizing spatial relationships, i.e., poor spatial skills.

This problem is significant in the context of engineering domain because ED is one of the fundamental courses for various engineering disciplines like Mechanical Engineering and Civil Engineering. Spatial skills, such as mental rotation (MR) play a major role in learning concepts that involve 3D visualization. Consequently, for the successful learning of such courses, students should be trained for the development of MR skill.

In order to address the issue, we have created a Blender-based training program ‘TIMeR’ to improve mental rotation skills of the students and hence the learning of relevant concepts. TIMeR stands for ‘Training to Improve Mental Rotation Skills’. TIMeR has three phases which are the preparatory phase, the training phase, and the transfer phase. Each phase has hands-on MR training task which is tightly coupled with the cognitive steps of mental rotation. Each task is executed using the instructional strategy of demo-drill-practice (DDP).

In this thesis, we present the work carried out in order to develop TIMeR and investigate its impact on the improvement of students’ MR skill. We also extended the work to the domain of engineering drawing and computer graphics. Total seven empirical research studies were carried out by applying mixed methods research design. The key results include (i) TIMeR found to be effective in the development of mental rotation skill, engineering drawing problem-solving performance and computer graphics problem-solving performance (ii) TIMeR helped students in resolving their engineering drawing learning difficulties (iii) TIMeR resulted as a workshop model and was successfully incorporated in engineering drawing course. The training structure would be useful for teachers to create their own mental rotation training program when the training objects are of three-dimensional nature.

The proliferation of information and communication technologies (ICT) has led to its widespread use in classrooms around the world in the last two decades. However for
improved student learning the focus of teaching-learning practice has to shift from routine use of ICT for demo/display to effective ICT integration, that is, the comprehensive process of applying ICT to the curriculum to improve teachinglearning, that relies heavily on pedagogical design. Teacher professional development (TPD) programmes that focus on pedagogy related to integration of ICT in classroom to inform effective teaching practices are one way of providing this solution.

Two key issues related to TPDs in the Indian engineering education context are: (i) Reliance on in-service in-service short-term training programmes (STTPs) and (ii) Issue of  large-scale. The number of in-service teachers existing within engineering education is around 0.5 million, introducing the need for scalable TPD programmes. Thus apart from the need for a good design, complexities may arise due to the scale. Thus the broad problem statement of this thesis is: How to improve the design and delivery of large-scale training programmes to in-service faculty in Indian engineering education, to enable them in effectively integrating Information and Communication Technology (ICT) tools within their teaching-learning context?

In order to address this problem, we have created the Attain-Align-Integrate-Investigate (A2I2) model for designing of technology integration training programmes. The A2I2 model has its theoretical basis on constructive alignment (Biggs, 1996), and it utilizes spiral curriculum (Bruner, 1960) and active learning (Prince, 2004) in its implementation. Design Based Implementation Research (DBIR) approach formed the methodological basis of this research. This model was used to design and implement five training programmes under the banner “Educational Technology for Engineering Teachers” (ET4ET) that got implemented across three different modes – face-to-face, blended online and massive open online mode. In line with the DBIR approach, evaluation studies conducted in each iteration informed us of the effectiveness of the training and also helped in refining the model. The evaluations were done on the metrics of reaction, learning, behaviour, participation rates while scaling and sustainability. Key results include (i) Participant teachers‟ evaluations were done on the metrics of reaction, learning, behaviour, participation rates while scaling and sustainability. Key results include (i) Participant teachers’ reporting attitude shift from teacher-centric to student-centric practices, (ii) Participants’ showing increased perception of competency in the use of wikis,
screencast and visualizations within their practice, and (iii) Medium-term sustainability of training benefits observed at the levels of teacher, student and institution. The iterative refinement of the A2I2 model also resulted in three design principles – Pertinency, Immersivity and Transfer of Ownership – that can be used to scale and sustain TPD efforts.

Learning Analytics is one of the focus areas to understand teaching learning practices in the current technology enabled learning era. The insights gained thereby assists to design better learning experience for students. We consider three levels to analyze learning data. A Micro level view focuses on an individual learner. A Macro level view focuses on the overall group of learners, for instance a class of students. A Meso-level views analyze cohorts (sub-groups) in those learners. In this thesis we have conceptualized Interactive Stratified Attribute Tracking (iSAT), as a meso-level visual analysis model for educational data. It helps to track transitions across time or across attributes of collected data and thus build a narrative about the learners or learning context as it changes across time or across attributes. I adopted Design and Development Research methodology to conduct the research in three phases. Initially in the Need and Context analysis phase, I established the need of analyzing transitions at a meso-level by stakeholders. I also reviewed the current learning analytics techniques for analysing cohorts and the limitations of existing dashboards and visualizations to support such an analysis. Based on those findings, I set the goal of design, development and evaluation. Next, in the Design and Development phase, I followed the Design Science paradigm to create iSAT model for visual cohort analysis at the meso-level. There were three stages in the design and development phase; (i) Genesis stage of the iSAT model, (ii) Refinement of that model and (iii) Implementing the iSAT model as a web-based free access tool. I defined the constructs of that meso-level analysis and evolved the methods involved in generating, representing and interpreting the information. iSAT model was applied by 12 researchers in 9 different scenarios to analyze their educational datasets, resulting in 12 peer-reviewed published research studies (6 conferences, 1 journal and in 2 thesis). Further, to proliferate the iSAT model and tool among stakeholders, we conducted 4 iSAT workshops in 3 international conferences and 1 in-house symposium. In the Evaluation phase, we synthesized the usefulness of iSAT model from the 12 case studies, studied
perception of first time users of iSAT and analysed applicability. iSAT helped both instructors and researchers to provide an overview of the transition patterns based on which they understand dynamics of the cohort and compare them. It can aid instructors in instructional decisions making and researchers to refine their analysis from the point of view of cohorts. The mean SUS score for iSAT tool was 71.57, indicating it as an acceptable system for analysis by the users.

Students should be able to correlate and apply the conceptual knowledge acquired from their classroom with the experiments carried out in laboratories. For this, students should be able to understand concepts and models at the ‘microscopic level’ (such as atoms or molecules), and link it to their corresponding observable / manipulable variables at the ‘macroscopic level’ (such as current and voltage). In this thesis, we refer to this as micro-macro thinking. It has been reported that students treat these two as disjoint sets, leading to lack of understanding of the complete system and difficulty in applying concepts to the real-world scenarios. The aim of this thesis is to design and develop a technology enhanced learning environment which will help students in developing this skill of micro-macro thinking.

In this thesis, we have designed, developed and evaluated MIC-O-MAP (MICroscopic Observations MAcroscopic Predictions), a TEL environment to develop students’ micro-macro thinking skills. The key pedagogical features and learning activities in MIC-O-MAP include a simulation of the microscopic world, prediction questions, justification box, real world answer for comparison and judgement, assertion and reasoning questions with dynamic feedback and dynamically linked multiple representations. These features have been included based on the pedagogical theories of inquiry learning, self-regulated learning, question prompts, types of scaffolds and methods of feedback and assessment. The entire interactive process of learning with MIC-O-MAP is mediated by a pedagogical agent which addresses the queries in the mind of the student and allows repetition of a task till mastery is reached. The environment is semi-open ended so that every student can have a unique learning path and can trace this path while interacting with the environment. Design Based Research (DBR) methodology has been followed for the design and development of the MIC-O-MAP TEL environment. Two iterative DBR cycles were implemented. Six MIC-O-MAP modules in analog electronics were developed. Eight research studies using explanatory sequential mixed method approach were carried out that included quasi-experimental studies (Ntotal= 249) and qualitative strands. The participants were students from first year of science and engineering from colleges affiliated to University of Mumbai, a large public urban university in India.

Results showed that students who work with MIC-O-MAP develop micro-macro thinking skills compared to a control group. Interaction analysis showed that there exists a contrast in the interaction paths of students who scored high versus low on a post-test on micro-macro thinking after interacting with MIC-O-MAP. The high scorers extensively use MIC-O-MAP features and go back and forth for sense-making, whereas the low scorers linearly progress through a MIC-O-MAP module to completion. We have also identified productive actions which are actions that enable learners to effectively use the features of MIC-O-MAP for achieving the objective of connecting the micro-world dynamics and the macro-world processes of physical phenomena.

Contributions of the thesis are: MIC-O-MAP TEL environment which has been evaluated in multiple studies, 6 MIC-O-MAP modules in the domain of analog electronics, productive actions to be used while interacting with MIC-O-MAP, the instructional design template of MIC-O-MAP for further module creation and identification of different learning paths of students while interacting with MIC-O-MAP.

Assessment is an integral part of instruction and it has a profound influence on what students’ study, how much they study and how effectively they study. The design of an
assessment instrument (AI) is one of the major components of assessment process. If students are subjected to random and unfair AIs, then assessment may not serve the intended purpose. Hence, the quality of any educational assessment exercise depends on the quality of AI used. Process of evaluation of AI quality brings lot of subjectivity into it as the benchmark can vary from person to person. Manual evaluation of quality by considering all the parameters is a very cumbersome task.

With this background, the broad research objective of the thesis is “How to improve the quality of AI and how technology will assist in this process? There are two ways to improve the quality of AI, namely (i) Evaluating the quality of teacher generated AI and providing feedback or (ii) Automatically generate the AI, so that the quality is ensured at the time of creation itself. The first approach is the major focus of my thesis. Pursuing these two approaches resulted in the design of two frameworks for the generation and evaluation of AI.

There are many quality parameters of AI. Based on literature, the measure of alignment of AI against the Learning Objectives (LOs) of the course is adopted as the quality of AI in our work. Implementation of frameworks resulted in the development of tools, namely, Instrument Quality Evaluator (IQuE), Teacher Training Module (TTM) and AI Generator (AIGen).

IQuE measures the quality of AI in terms of its alignment with the LOs of the course. An ontology based Knowledge Representation (KR) mechanism is designed to integrate the contents of syllabus, LOs and AI. Content and cognitive level information are extracted from LOs and questions using simple Natural Language Processing (NLP) techniques. Measure of alignment is formulated based on the commonalities and differences in concepts covered and cognitive levels from LOs and questions respectively. IQuE provides two types of outputs; a numerical measure of alignment and its visual representation. It also estimates the utility of each question indicating its contribution towards LOs. Accuracy of IQuE was tested with large number of samples (N=1000) and the accuracy with respect to content and cognitive level alignment are 91.2% and 93.23% respectively.

Using IQuE, a TTM is developed that can be used to train teachers to write good assessment questions against given LOs. It has a multistage environment and is supported by a formative feedback mechanism that gives the feedback about the alignment of teacher written questions against a system displayed LOs.

We have built a prototype version of AIGen that facilitates automatic generation of AI from the teacher entered AI specification (AIS) and tagged question repository. In the
preliminary investigation, it was found that the generated AIs were 80% compliant with the corresponding teacher entered AIS.

The context of the research work is AIs designed for written examinations in a typical university scenario in engineering curriculum. All the samples for the study are taken from the Data structures course in engineering curriculum.

The problem of improving and assessing programming comprehension and debugging skills through self-learning spoken tutorial workshop methodology is studied in this work. We analyzed the performance of 220 participants who volunteered for Java self-learning workshops and 180 participants from traditional Java class for this research study. Both the groups were from non-CS background undergraduate courses. We then examined the learner perceptions through standardized self-regulated learning questionnaire and determined the correlation with actual performance through a post-test. We found significant differences among the workshop and classroom groups in terms of perception and performance on Java post-test. The mean average scores for workshop and classroom groups were 69% and 63% in Java post-test. Based of these findings, we further investigated the differences among the conventional classroom methods and spoken tutorial workshop methodologies through qualitative and quantitative studies. These results are in agreement with literature. We improved our research design to determine its effectiveness in improving basic and advanced Java programming skills. We have found that most of the students attempt to memorize typical program code instead of writing a fresh code applying the programming concepts. In order to address this difficulty, we used the Program Chunking Technique as a instructional methodology. We conducted a research study with two groups of 80 non-CS background participants and investigated their performance in Java workshops. We found significant improvement in syntax and semantics of basic Java programming concepts. Program chunking techniques could not address advanced Java concepts such as recursion. We incorporated the Jeliot Java tool to improve performance on advanced Java concepts. We examined two groups of 40 non-CS background undergraduates and assessed them through assignments and post-test on advanced Java concepts. We found significant improvement in experimental group on conceptual understanding and post-test performance, while the control group did not show any such improvement in performance. Notably, students with misconceptions had an opportunity to correct and improve their programming skill through visualization. We improved our research design by incorporating diagnostic feedback along with visualization based on the student feedback. We examined two groups of 30 participants through this study and found that experimental group showed significant improvement in advanced Java concepts as compared to control group participants. The mean average scores of experimental group on comprehension and debugging skills are 84% and 82% respectively. The corresponding numbers for the control group are 79% and 78% respectively. The CohenD effect size among the groups is 0.68 which indicates that difference among the groups is significant. Improvements made to self-learning workshops through these research studies have helped students improve their basic and advanced Java programming skills. The main contributions of this thesis are:

1. Validation of the effectiveness of self-learning spoken tutorial workshops through large controlled experiments.
2. Validation of program visualization tools and spoken tutorials.
3. LMS Moodle based assessment system to evaluate programming skills.
4. Guidelines for conducting self-learning programming workshops.

Keywords: Self-learning workshop, Spoken Tutorial, Programming comprehension, Debugging, Program chunking, Visualization

Educational visualizations (animations/simulations) have the potential to achieve important learning objectives, especially in science and engineering domains. But these
objectives remain unrealized when instructors simply lecture with the tool, which is one of the popular strategies used worldwide. This inability of instructors to create student-centred learning designs (LDs) that exploit visualization affordances, has been identified as an important barrier to effective integration of information and communication technologies (ICT) like visualization. It becomes more challenging for instructors teaching in instructormediated classrooms that are common in India and the developing world. In these classrooms students’ interaction with the visualization is necessarily mediated via the instructor. Thus the specific problem targeted in this thesis is instructors’ inability to design effective learning activities with visualization. The target user population is tertiary level instructors who are novice designers and teach in instructor-mediated classrooms.

We have developed the Customized Visualization Integration System (CuVIS) framework for science and engineering instructors. The CuVIS framework targets four design
impediments identified from our studies with instructors - i) operationalizing constructive alignment and ii) meaningful learning, iii) framing group activity questions (conceptual level) and iv) designing implementation of active learning strategies (implementation level). CuVIS framework presents the conceptual level guidelines through the Activity Constructor prompts. They guide instructors to take design decisions at appropriate points in the LD creation process. Each prompt contains guidelines with illustrative examples from the instructor’s domain. The implementation level guidelines are provided through the LD Blueprint outlining the classroom implementation design. Both the prompts and the blueprint vary with variation in objective and activity time duration specifications. CuVIS framework prototype was tested iteratively with instructors in four cycles. Effectiveness of CuVIS framework has been shown through its positive impact on instructors’ design expertise in terms of: (i) their TPACK levels and (ii) their pedagogical practice. Also, implementing CuVIS LDs have led to successful achievement of the chosen objective through student post-test results. Usefulness and usability of CuVIS tool, built as a digital interface to the framework, has been established through large scale survey with 1200 + instructors.

This thesis is organized as follows: Chapter 2 presents problem space literature followed by existing solutions like teaching principles, best practices portals, teacher training
programs and LD frameworks and tools. Gap analysis of existing solutions in our research context leads to our research objective of developing a framework to create constructively aligned, meaningful, customized LDs for teaching using visualization in instructor-mediated classrooms. Chapter 3 presents the research methodology of Design and Development Research (DDR) that was chosen to build such a framework. DDR was deemed suitable since it supports creation of solutions to a real-life problem through iterative prototype testing with the target users who are co-participants in the design process. It proceeds through three phases of Problem Analysis, Design and Development and Summative Evaluation – each of which is described in detail for CuVIS framework in Chapters 4, 5, 6 and 7. Chapter 8 documents development and evaluation of CuVIS tool. Chapter 9 discusses implications and limitations of CuVIS framework and Chapter 10 presents the contributions and future work.

Interactive learning environments (ILEs) are computer-based simulation environments which allow learners to interact with the learning material using various interaction features. The varied levels of interaction offer varied learning experiences and learning outcome from ILEs. While ILEs have shown potential for improved learning in various domains, empirical studies have shown mixed learning results. Particularly, studies have shown that the interactive nature of ILEs could not always lead to better learning. On this background, the broad research objective of this thesis is: 'Under what conditions do ILEs lead to effective learning?' The context of study is a course on 'Signals and Systems', a foundational undergraduate course in Electrical Engineering.

The research issue was addressed by examining, analyzing and re-designing learning-conducive interaction features in ILEs, which would offer the required cognitive support to learners while learning from ILEs. We proposed 'Interactivity Enriching Features (IEFs)', which are additional interaction features needed to unleash the learning potential of ILEs. The overall solution approach included establishing the need for IEFs, identifying and designing of IEFs for variable manipulating interactions, investigating learning effectiveness of IEFs and exploring effect of IEFs on learners' cognitive load. As a part of thesis work, four IEFs were designed: Permutative Variable Manipulation, Productively Constrained Variable Manipulation, Discretized Interactivity Manipulation, Reciprocative Dynamic Linking.

Five research studies using explanatory sequential mixed method approach were carried out that included quasi-experimental studies (Ntotal= 437) and qualitative strands. The participants were students from second year of engineering from colleges affiliated to University of Mumbai, a large public urban university in India. The assessment instruments were designed to address the requirements of engineering curriculum and focused on 'understand' and 'apply' cognitive levels and 'conceptual' and 'procedural' types of knowledge within the chosen topics. Qualitative data in the form of screen captures and semi-structured interviews were used wherever needed as per the research design.

The results showed that higher level of interaction need not necessarily lead to higher learning but depended on the cognitive level and type of knowledge of the content. The findings provided evidence for the inclusion for IEFs to enhance learning from ILEs. The findings showed that learners learnt better with IEFs and thus, the need for strategic designing of interactions to meet learning demands of learners was emphasised. The improvement in germane load of learners could confirm the role of IEFs in offering the required cognitive support to learners that led to improvement in learning.

The major contributions of the thesis are: determining the IEFs for effective ILEs, designing of four IEFs for content manipulation interactions, recommendations in the form of Interactivity Design Principles based on the findings of empirical studies conducted to test effectiveness of IEFs, and development of Interactivity Enriched Learning Environments for three different topics in Signals and Systems.

Students, who study in their primary language in K-12 and go on to do their undergraduate education in English, are known as vernacular medium students. Vernacular medium students face difficulty in acquiring programming knowledge in English medium of instructions (MoI). Solutions targeted towards improving their English proficiency take time while continue teaching in primary language MoI limits the students’ ability to compete in a global market. The key challenge is in developing a framework that helps vernacular medium students to comprehend the educational content presented in English MoI. It will not only help them to develop content knowledge but increase English competency also.

In this thesis, we address the problem of primary language learners in learning computer programming in English MoI. In our solution approach, we first identify the problems of vernacular medium students from the literature review and then reconfirm them in Indian context using a qualitative study. We identified and tested language based scaffolds, cognitive scaffolds, and the environment in which these scaffolds work.

This thesis presents five research cycles which were used to identify, select and test the effectiveness of various scaffolds to teach programming to vernacular medium students. The research cycles produced five different prototypes. Each prototype use different set of scaffolds in learning material, learning environment and presentation. Learning material for each prototype was selected or designed to reduce the cognitive load of students and provide language-based scaffolds. We used two different educational environments, 1) classroom environment and 2) self-paced video-based learning environment to test our prototypes. Visualization guidelines to teach various content types and multimedia principles are followed to reduce the cognitive load of students.

Cognitive scaffolds to reduce intrinsic cognitive load are identified from instructional design principles and visualization guidelines to teach various educational content types. We found that instructional design principles help in writing learning objectives and decide prerequisites. Use of instructional design helps in the systematic planning of instructions that removes instructional gaps and help learners to comprehend the presented learning material by reducing the intrinsic cognitive load. Visualization guidelines to teach various educational content types (e.g. fact, process, concept, procedure, and principle) helps in design instructions that reduce the intrinsic cognitive load of vernacular medium students. We selected and tested multimedia principles that reduce the extrinsic cognitive load of vernacular medium students. These principles are split-attention effect, segmentation, pre-training, synchronization, redundancy effect, verbal redundancy and attention cueing.

We identified several language based scaffolds that reduce the mental effort of a vernacular-medium student that are used to translate the educational content presented in English only MoI. These language-based scaffolds are 1) Use of simple English MoI, 2) Explain semi-specialized and specialized words on the first occurrence, 3) Use of slow pace for vocal explanation. We also identified language-based scaffolds for bilingual MoI when classroom based environment is used, these scaffolds are 1) Use of simple Hindi MoI for vocal explanation, 2) use of code-switching, 3) Use of English MoI for specialized and semi-specialized words.

We conducted two qualitative studies, and three quantitative studies to measure the effectiveness of various scaffolds. We used classroom based environment in two research cycles and self-paced video-based learning environment in three research cycles. We find that self-paced video-based environment is more suitable for vernacular medium students than a classroom environment if English only MoI are used.

The main contribution of this thesis is 1) identification of language-based scaffolds that help in comprehending the educational content presented in English only MoI and bilingual MoI 2) a framework that helps teachers to plan instructions to teach vernacular medium students based on various conditions 3) the selection of visualization guidelines and multimedia principles to provide cognitive scaffolds.

Engineering is a practice-driven profession. Engineering graduates should be able to demonstrate and apply thinking skills in addition to their domain knowledge. Engineering design thinking skill is one such important thinking skill. Even though this skill is being taught using various instructional methods such as project based learning, it is reported that students are unable to demonstrate engineering design thinking skill. A key challenge is in defining what to teach as engineering design thinking, and how to assess this skill.

In this thesis, we address the problem of developing and assessing engineering design thinking skill among undergraduates. In our solution approach, we operationalized engineering design thinking skill in terms of measurable competencies. We identified the following engineering design competencies: Structure Open Problem, Multiple Representation, Information Gathering, Convergent Thinking and Divergent Thinking. We developed rubrics as a formative assessment instrument for these competencies. The rubrics assess students’ progress of competency acquisition as well as provide constructive feedback to attain competency in a given design task.

To help students attain the engineering design competencies, we designed TELE-EDesC - Technology Enhanced Learning Environment for Engineering Design Competency. TELE-EDesC is a self-learning environment which includes interactive learning activities, referred to as ‘Learning Dialogs’. TELE-EDesC Learning Dialogs harness the affordances of modern technology such as interactive experimentation, self-regulation, and personalized feedback, to trigger essential metacognitive processes required for engineering design thinking.

We developed TELE-EDesC learning modules for Structure Open Problem (SOP) competency for topics in analog electronics, and tested them using quasi-experimental studies (N=295) as well as qualitative interaction analysis, with second year engineering students. We found that TELE-EDesC was effective for learners in attaining SOP competency (statistically significant differences, p<0.01). From the interaction analysis, we identified productive learning behaviours of successful students and revised TELE-EDesC to promote such behaviour among all learners.

The main contributions of this thesis are: TELE-EDesC learning modules that have been empirically validated for SOP competency for a range of topics in analog electronics, a pedagogical framework to develop TEL environments for engineering design competencies, and assessment rubrics for engineering design competencies.

The present thesis entitled “The use of concept maps and an instructional design model in the classroom teaching of thermodynamics and Internal combustion engines” is organized into thirteen Chapters. Currently, present engineering education system makes use of diverse kinds of instructional methods and innovative teaching methods for teaching engineering concepts (Cuban, 1986). Such implementation of innovative teaching methods and affordable technologies are employed with the expectation of reforms in engineering education. Even after spending at least four years in the undergraduate programmes, a majority of students do not develop a good understanding of the basic principles involved in the theory of engineering as well as applying the learning in real life situations. In mechanical engineering curriculum, thermodynamics and internal combustion engines are core subjects and student are facing difficulties in understanding concepts in these subjects (Mulop et al., 2011, Bahr, 1999, Huang and Gramoll, 2004, Anderson et al., 2002, 2005). The present work will elaborate on the current problems in engineering education and the specific difficulties in teaching thermodynamics and internal combustion engines. A solution to the problem will only emerge if the learning processes are strengthened by addressing all major components that go into the process of learning.