Enhancing students’ understanding of vectors through personalized adaptive learning

In introductory college physics courses, it is important that students have a thorough understanding of vectors since several topics depend on vectors. When the test of understanding of vectors (TUV) was administered to the students completing the algebra-based College Physics I course, it was found that students had considerable difficulties in certain vector concepts. To overcome these difficulties, the question-bank for personalized adaptive learning (PAL) was modified. These modifications were implemented in a later semester for the students of the same course. In the current work, we present students’ performance on 12 questions on TUV for two semesters with a focus on improvement in students’ performance on graphic representation and determination of direction and magnitude of a vector presented in unit vector notation, understanding of graphical methods for vector addition, subtraction and multiplication of a vector by a negative scalar. This study presents the development of questions in PAL assignments which may be beneficial for the teaching community.


Introduction
In introductory physics courses, several topics depend on vector algebra and hence it is important that students have a thorough understanding of vectors. When the test of understanding of vectors (TUV) [1] was administered to the students in College Physics I course, the results confirmed the need for new instructional material to foster students' understanding of vectors. With this goal, we referred to McDermott's study [2], and followed the process for developing a new curriculum. The process has three steps: (1) conduct research on students' understanding, (2) develop curriculum based on these research findings, and (3) investigate the effectiveness of the new curriculum by carrying out an assessment.
The literature objectively reports that the use of personalized adaptive learning (PAL) is an effective teaching strategy [3,4]. In the present work we report the development of variety of questions for PAL based on data-driven approach to enhance students' conceptual learning and mathematical skills which facilitated students' improved performance on several concepts, such as, (a) calculating the magnitude and direction of a vector written in unit-vector notation, (b) using graphical methods for representing the vector written in unit-vector notation, (c) graphically representing the resultant vector for a vector multiplied by a negative scalar, and (d) graphical representation of resultant vector for vector addition in 2D, as well as for subtraction of vectors in 1D and 2D. The objectives of this study are, to present evidence of the need for modifications in PAL content, to present the strategies to design questions for PAL assignments and evaluate their effectiveness. The general research question of this study is: how effective are PAL assignments in students' understanding of vectors?
One of the authors of this article has published several studies related to the conceptual difficulties that students have in the subject of vectors [e.g. 5,6]. He has also designed research-based multiple-choice tests on this topic [1,7]. In addition, he has also been involved in studies where tutorial worksheets have been designed to enhance students' understanding of specific vector topics [e.g. 8]. This is the first study that the authors of this article present the design of PAL assignments to improve students' understanding of various vector concepts.

Previous research on students' difficulties with vectors
Several studies have been carried out on students' understanding of vectors. Knight developed vector knowledge test given to calculus-based introductory physics students to check if students have minimal knowledge of vectors to proceed with the study of Newtonian mechanics [9]. Knight studied the qualitative as well as quantitative knowledge of students for vectors. A study [10] conducted on 11th-grade students suggested that most students had difficulties in graphical and analytical vector addition, vector multiplication and determining magnitude and direction of vectors. Numerous incorrect preconceptions among high school students about the role and significance of the velocity vector components have been reported [11].
Studies conducted on students beginning their second semester of calculus-based and algebra-based physics courses in sequence, have shown, respectively, that more than one quarter and more than half of the students were unable to perform two-dimensional vector addition [12]. Also, students enrolled in a first-semester (Mechanics), or second-semester (Electricity and Magnetism) introductory calculus-based physics courses showed lower performance on vector addition and subtraction when vectors are posed as arrows instead of in algebraic notation (using i, j, k) [13]. The context, such as displacement, force, and no physical context has an influence on the representations used when sketching the two vectors to be added and on the vector sum [5].
The students who completed a calculus-based type course of mechanics have been found to have serious difficulties sketching the unit vector in the direction of a vector while using cartesian coordinate plane [6]. Additionally, for problems involving unit-vector notation, these students made frequent errors in calculating the direction and magnitude of a vector, vector sum and subtraction, multiplication of a vector by a scalar, and dot and cross product [1]. As an initial general survey, when TUV was administered to the first-year medical students to test the conceptual knowledge of vectors [14], an average score of 26% showed the weak overall performance of the students.

Personalized adaptive learning
PAL was implemented using Realizeit platform [15] and refers to online instruction that provides personalized learning experience [3,4]. PAL designed by the course instructor is based on how much the student has understood, on what points the student is not comfortable and what are the major gaps; the instructor then objectively worked on what the student needs to learn. To start the learning cycle, PAL is constructed as an appropriate learning pathway for the students. The grading formula uses two schemas, keeping the higher one: (1) 50% is awarded for completing all activities and 40% for the average mastery achieved (students' answers to the 'Check your Understanding' Questions), and 1% for each practice or revision activity completed (up to a maximum of 10%, which benefits those who may not have a perfect score the first time through, but go back and practice to get a better understanding/ higher score). And (2) 50% for completing all activities and 50% for the average mastery attained. This benefits those who may not have needed to go back and practice because they scored very well to start.
Each PAL module consists of two parts: the reading content followed by an assignment where students answer a variety of questions based on the reading content. The questions are developed based on data-driven approach. PAL assignments consist of questions in several formats, such as: multiple-choice, sketching the vectors using widgets and entering the numerical values for the answer. Examples of these questions are given in figure 1.
The problems (shown in figure 1) have randomized numerical values. To avoid guesswork from the students, each multiple-choice problem has eight answer choices. If a student submits an incorrect answer, the PAL system shows the correct answer providing immediate feedback. Students have the option to flag the question if they have difficulty, helping timely intervention by the instructor to facilitate learning.
Every time the student logs in to work on the assignment, the PAL system picks questions from the question bank designed for that particular vector section facilitating students to work on a variety of problems. Implementation of PAL is aligned with Knight's suggestion [9] that students must practice vector math and reasoning, and that additional vector problems should be assigned for several weeks after they get the feedback from their first vector homework.
Students can work on the PAL assignments as many times as they want, and these assignments are open till the day of the final exam. For their multiple attempts to work on the assignments, the highest score is registered in the grade book to be counted towards the overall grade in the course. Students also come across vector problems in later chapters covering topics, such as projectile motion, force, momentum, and equilibrium. Knight also emphasized on providing basic practice for vector math via computer-aided instruction [9] where an endless variety of problems could be provided along with immediate feedback.

Design of the modification in PAL
In the present work, we focus on modifications made to the PAL assignments to facilitate students overcome the difficulty of several aspects of vectors. Table 1 shows the modifications and their justifications. Determining the components of a vector and understanding how components are used to determine the magnitude and direction of a vector, is most basic to understanding vector math. Students must not just pay attention to the numerical value of the angle the vector is making but also understand relative to which axis the vector is making this angle. This could help them relate the direction of components to the quadrant the vector has to be drawn in. Questions on determining components based on vector widgets were used to facilitate students learn graphical representation of vectors.
Variety of questions on a graphical representation of vectors were added as multiplechoice questions. Questions on vector addition and subtraction were included where the resultant vector had to be determined using graphical methods. For making students familiar with unit-vector notation, the vectors were given in unit-vector notation and asked for the direction and magnitude of the resultant vector when these vectors were added or subtracted. Questions on vector addition using component method were added where the vectors to be added were in different quadrants. Graphical and mathematical questions on multiplication of vectors were added which also included widgets where students had to draw the resultant vector. In several questions students had to determine the direction of the resultant vector relative to different axis. Some example questions from the PAL question-bank which were added for the experimental group are given in the supplementary material and discussed in the latter sections.

Methodology
TUV, a widely used test, was administered to the students in College Physics I course at one of the largest research universities in the USA. College Physics I is an algebra-based introductory physics course which covers topics mainly from mechanics and waves. This course is offered primarily for students majoring in information technology, the biological sciences, and pre-health professions. This is also a pre-requisite course for the students taking Medical College Admission Test (MCAT).
Students' performance on 12 questions on modified version of TUV was analysed for the Fall 2019 and Spring 2022 semesters, which we will refer to as the control group and the experimental group, respectively. Both groups were not during covid and were face-to-face classes. The control group had 66.67% female students and 33.33% male students. The average age of students in the control group was 21.86. The experimental group had 77.78% female students and 22.22% male students. The average age of students in the experimental group was 20.30.
For both the groups, there were 2 h 50 min class sessions, twice a week. The physics curriculum was same for both groups. For both the groups, the topic of vectors was taught in class in exactly the same way. Both the groups were taught in face-to-face studio-mode class format [16]. Students were assigned to a group. Each group consisted of 3 students. Students worked on in-class activities, such as, working on worksheets consisting of problem-solving and concept-questions, experiments, and in-class student presentations were encouraged. Studio-mode promotes peer teaching and learning. Both the groups utilized PAL, except that experimental group utilized modified version of PAL, described in 'Design of the Modification in PAL' section.
Knight's [9] suggestion of students' active participation and getting engaged in group discussions to enhance students' understanding of vectors could easily be followed. The course was taught by the same instructor, and OpenStax book [17], which is free of cost, was adopted for the course.
As an in-class activity, students worked on two worksheets on vectors. The first one consisted of questions on drawing the negative of a vector and graphical addition of vectors. The second worksheet consisted of free-response questions for analytically determining

Modifications in PAL questions Justifications
Vector magnitude: For determining the magnitude of a vector, we included questions where the vectors were presented in unit vector notation and were also graphically represented using widgets. We also included questions where students had to determine the magnitude of a resultant vector using addition or subtraction component method.
• To promote the understanding of the magnitude of a vector in different scenarios, such as for single vector, or addition/subtraction of two or three vectors using component method.
• To enhance the understanding of the magnitude of a vector represented in unit vector notation.
Vector direction: For determining the vector direction, we included questions using several widgets, with vectors making angles relative to different axis. We also added questions where direction of resultant vectors in the answer choices for multiple-choice questions were presented relative to +x or −x axis and as clockwise or counter-clockwise from +x axis. We also included several questions where in the question the vector directions were presented such as, south of east and south of west, east of north and west of north. Several questions were included where vectors were represented in unit vector notation.
• To enhance the understanding of the vocabulary for vector directions, such as counter-clockwise or clockwise relative to an axis. Also, to promote the understanding of vector direction when wordings such as south of east or south of west etc are used. • Finally, to enhance the understanding of vector direction for a vector represented in unit-vector notation.
Vector multiplication: We included questions where the vectors were presented in graphical and word format and had to be multiplied by a positive or negative scalar.
• To promote the understanding of the change in vector magnitude and direction due to multiplication. • To enhance the graphical understanding of the change in magnitude of a vector due to multiplication. • To promote the graphical understanding of the change in the direction of a vector when a given vector is multiplied by a negative scalar.
Vector addition and subtraction: We included a variety of questions for graphical addition and subtraction of vectors. We added questions where students could use widgets to draw negative of a vector, vary the length of a vector, and graphically add and subtract vectors using widgets. We also included multiple-choice questions for this topic.
• To promote the understanding of head-to-tail method of vector addition. • To promote the graphical understanding of vector addition and subtraction in 1D and 2D.
magnitude and direction of a vector, representing the vector in unit-vector notation and addition of vectors using component method. To make students pay attention to the direction and relate the magnitude to the length of the arrow representing a vector, students had to draw the vectors and its components. For both groups, students participated in in-class presentations to present their work on the worksheets, to exchange ideas with the entire class, learn different approaches to solve the same problem and to reflect on the errors. Students were administered TUV as a pre-test, during the first week of the classes before the topic of vectors was discussed in the class and as a post-test during the last week of the classes.

Evidence of the need for modifications in PAL
To present evidence of the need for modifications in PAL content, we analysed the post-test data for the control group. Table 2 presents the post-test data for the control and experimental group.
When TUV was administered to the control group, the results indicated at students' difficulty in certain vector concepts. Students' performance on calculation of direction and magnitude of a vector written in unit-vector notation was 29% and 41% respectively. Only 27% students could correctly answer the question on graphical addition of vectors in 2D. For graphical subtraction in 1D and 2D, and graphic representation of a vector multiplied by a negative scalar, the performance ranged from 13% to 19%. These results present evidence of the need for modifications in PAL.

Effectiveness of the modifications in PAL
To evaluate effectiveness of the modifications in PAL assignments, we compared the scores on the modified version of the TUV after instruction for the control and experimental groups. In the control group, 93 students participated in the pre-test and 63 students participated in the post-test. In the experimental group, 88 students participated in the pre-test and 69 students participated in the post-test. We first evaluated whether both the groups were comparable by analyzing the scores on the pre-test. Since the distribution of scores were not normal, we decided to use the non-parametric Mann-Whitney U test [18]. This test indicated that the pretest scores of the students in the control group (Median = 3) did not differ significantly from those of students in the experimental group (Median = 3), U = 3983.5, z = −0.31, p = 0.76. U is the Mann-Whitney U statistic. Based on this result, we can state that both groups were comparable.
For comparing the students' performance for the two groups after instruction, we decided to use the Mann-Whitney U test [18] since the distribution of scores on post-test for both the groups were not normal. For post-tests, average scores for control and experimental groups are 5.14 and 7.96, respectively. Standard deviations for control and experimental groups are 2.52 and 2.75, respectively. This test indicated that the scores obtained by students in the experimental group (Median = 8) were significantly higher than those obtained by students in the control group (Median = 5), U = 968.5, z = 5.49, p < 0.01. This result shows that the modifications in the instruction helped students enhance their understanding of vector concepts.
As can be seen from table 2, on several questions we found a higher percentage of correct responses for both the groups in the post-test. Students showed better understanding of graphic representation of x and y components of a vector (Questions 2 and 4) with, on average for both the groups, about 86% and 81% correct responses, respectively. For both the groups, 57% students could correctly calculate x-component of a vector (angle measured from y-axis). For the question on comparing the vector sum's magnitude of two same-magnitude vectors at For the vector concepts, mentioned in table 2, pertaining to calculation of direction and magnitude of a vector written in unit-vector notation, graphical addition of vectors in 2D, graphical subtraction of vectors in 1D and 2D, and graphic representation of a vector multiplied by a negative scalar, we see a significant increase in the correct answer in the post-test for the experimental group. We collapsed the wrong options, and using Chi-Square Test we found a significant difference in the distribution of answers for both the groups with p < 0.01. In the following sections, we focus on discussing students' difficulty and steps taken for remediation for questions pertaining to these vector concepts. (questions 10 and 12). In this section, we discuss students' performance on determining the direction and magnitude of a vector written in unit vector notation and modifications to the PAL assignment. Unit-vector notation is an important topic and is covered in several books used to teach algebra-based introductory physics courses.

Direction and magnitude
It is important to note that while students' performance on graphic representation of a vector written in unit-vector notation (question 5) for control and experimental group was 62% and 83%, respectively, in the control group students had difficulty answering questions on determining the direction and magnitude of a vector written in unit-vector notation. For the experimental group, in PAL assignments, several questions were added where students came across a variety of ways a vector direction can be represented as well as questions where the vectors were represented in unit-vector notation. PAL assignments included several questions on determining the magnitude of a vector using Pythagorean theorem.
For question 10, a vector is given in unit vector notation (A = −3i+4j) and asks for determining the direction of this vector as measured from the positive x-axis. Students' response to this question depends on (a) their understanding of unit-vector notation and (b) using the inverse trigonometric function to determine the angle as well as being able to represent the direction relative to the positive x-axis. Students' response to this question is given in table 3. The correct answer for this question is 126.87°. For the experimental group, students' performance on this question improved by 25% with 54% students answering the question correctly. It is important to note that students' performance on this question is same in calculus-based Electricity and Magnetism course [1] which is the second course in the sequence for introductory calculus-based courses offered primarily for physics and engineering students.
In the control group, the most frequent incorrect answer students chose was the option 53.13°, with 24% students choosing this option. Thus, these students used the inverse tangent function correctly but made a mistake in representing the direction of the vector as measured from the positive x-axis. For the experimental group, this error reduced with only 11% students choosing this option.
Looking at students' difficulty in the control group, multiple-choice problems were included in PAL assignments to facilitate students' learning of unit-vector notation and to familiarize students with different ways of representing the vector directions. Questions were added where two vectors were given in unit-vector notation, and students had to determine the direction of the resultant vector when these two vectors were added or subtracted (questions 1 and 2 in the supplementary material). To determine the resultant vector students had to first understand how vector components are used to represent a vector in unit-vector notation. For finding the direction of resultant vector, students had to determine the magnitude and direction of the components for the resultant vector and then use an inverse trigonometric function. Following this, students had to correctly represent the direction relative to the positive x-axis.
It is a common problem that students often do not pay attention to how the vector direction is asked for. Hence, even though students use the inverse trigonometric function correctly, they give incorrect responses to the vector direction if the direction is asked relative to a different axis. It is important to make students learn variety of ways in which vector directions can be represented. Hence, questions were designed where vectors in the problems were presented relative to +x and −x axis, and relative to +y and −y axis (example questions 7, 9 and 13 in the supplementary material).
Students also came across questions where the vector direction was presented, such as south of east and south of west, east of north and west of north (question 5 as an example in the supplementary material). Also, vector direction in the answer choices for several multiplechoice questions were given relative to +x or −x axis and as clockwise or counter-clockwise from +x axis (question 1 and 8-10, in the supplementary material).
For question 12, a vector is represented in unit-vector notation (A = 2i+2j) and asks to determine the magnitude of this vector. For this question, in addition to understanding the unit-vector notation, students had to use the Pythagorean theorem to determine the magnitude of the vector. Table 4 gives students' response to this question. Correct answer to this question is 8 . While about 40% students could answer this question correctly in the control group, in the experimental group 80% students could correctly answer this question. Using Chi-Square Test. In the control group, the most frequent incorrect answer students chose was the option of magnitude being 2, with 27% students choosing this option. In the experimental group, since 80% students chose the correct answer, the most frequent incorrect answer was very low with 6% students choosing the options of magnitude 2 and 4, each.
Questions on vector addition and subtraction (example questions 3 and 4 in the supplementary material) were added in PAL assignments where two vectors were given in unit vector notation and students had to determine the magnitude of the resultant vector. Students had to first determine the components of the resultant vector and then use these components to determine the magnitude of the resultant vector using Pythagorean theorem. In addition to several questions where the vectors were represented graphically, several questions were added to PAL assignments where vectors were explained in text (example questions 5, 6, 8-10 in the supplementary material) and asked for determining the magnitude of the resultant vector. The improvement in students' performance on the questions can be attributed to better understanding of vectors represented in unit-vector notation and coming across variety of problems where magnitude of resultant vector had to be determined. Question 5 asks for the graphic representation of, −2i+3j, a vector given in unit vector notation. Table 5 gives students' response to this question. Correct answer to this question is option B where the vector is drawn in the second quadrant correctly showing the magnitude and direction of the components. Students' performance in this question improved by 21%, with 83% students in the experimental group answering this question correctly. Most frequent error in the control and experimental group was students choosing option A which shows the vector in the first quadrant with incorrectly showing the x-component as positive. While 22% students in the control group chose this option, this error reduced in the experimental group with 11% students choosing this option. Looking at students' performance in control group, several questions were added in the PAL assignment where students had to draw the vectors using widgets.
In experimental group, students were more likely to answer this question correctly due to being more familiar with unit-vector notation and additional widgets added for drawing the vectors. Students also came across several questions where they had to find the components of vectors oriented in different quadrants (example question 7 given in the supplementary material) which could have helped students relate the components to the quadrant in which the vector had to be drawn in. 5.2.3. Graphical vector addition (question 1). Students' performance on question 1 for graphical representation of resultant vector for addition of vectors in 2D, is presented in table 6. In this question, the two vectors which are to be added, A = −3i+3j and B = −2i −2j, are placed tail-to-tail. The question asks for graphical representation of the resultant Table 4. Students' response on question 12. The correct answer for this question is option B. X 2 (1, N = 132) = 20.525, p < .01.

Option Answer choices Control group Experimental group
8% 4% E 8 6% 4% Table 5. Students' response on question 5. The correct answer for this question is option B. In answer choices A, B and C, the tail of the vector is at the origin of the coordinate system, while in D and E, it is at (−2, +3) and at (−2, 0), respectively. X 2 (1, N = 132) = 7.117, p < .01.

Option
Answer choices (Graphically represented) Control group Experimental group vector when these two vectors are added. To answer this question, students are supposed to understand the head-to-tail method of vector addition. The answer choices in table 6 are presented in unit vector notation which were graphically represented in the question. Students' performance on this question improved by 22% with about 50% students answering the question correctly in the experimental group. For both the groups, most common error remained the same with 40% and 36% students in control and experimental groups, respectively, choosing option D which shows the resultant vector being drawn from the head of vector B to the head of vector A, and hence not following the head-to-tail method of vector addition. There was a 10% decrease in students choosing option A where the ycomponent was represented correctly but the x-component had an incorrect magnitude.
Looking at students' difficulty in head-to-tail graphical addition of vectors, several questions in PAL assignments were added for graphical addition of vectors, which included questions where two vectors to be added were represented in 2D but not placed tail-to-tail (example Question 11 given in the supplementary material), a vector in 1D had to be added to a vector in 2D while these two vectors were placed tail-to-tail (example question 12 given in the supplementary material), and vectors in 2D with tails placed at the origin (example question 14 given in the supplementary material). Several variations to these example questions were included, as mentioned in the supplementary material, to emphasize on headto-tail method of vector addition. 1D and 2D (questions 11 and 7). In this section, we present students' performance on questions 11 and 7, which are on graphical subtraction of vectors in 1D and 2D respectively. Table 7 presents students' response on the question 11 on graphical subtraction of vectors in 1D. The answer choices in the table are presented in unit vector notation which were represented graphically in the question. In the question, two vectors, A and B are represented graphically. A has a magnitude of 3 units and is pointing to the left. B has a magnitude of 5 units and is pointing to the right. The question asks for the resultant vector for A-B.

Graphical vector subtraction in
Students' performance in the experimental group improved by 50% compared to the control group. The most common error in control group was students choosing option B which is actually A+B. This reflects a common misconception among students that if two vectors are in opposite directions then they are not required to reverse the direction of the vector which has to be subtracted, that is, to first find the negative of the vector to be subtracted and then do the vector addition. For this question, students could have first reversed the direction of B to get -B and then added it to A. This error reduced with 51% less students choosing this option in the experimental group. Table 6. Students' answer on question 1. The correct answer is option E. X 2 (1, N = 132) = 6.902, p < .01.

Answer choice
Answer choices (graphically represented) Table 8 presents students' response to the graphic subtraction of vectors in 2D [1]. Students' performance on this question improved in the experimental group with 52% students answering the question correctly compared to control group where only 17% students answered this question correctly. Most common error in the control group was students choosing option D which reduced by 34% for the experimental group.
For these two questions, looking at the students' performance in control group, the PAL question bank was modified to have students understand the meaning of negative of a vector. Students came across several questions where they had to draw the negative of a vector using widgets, choose the correct option for multiple-choice questions where they had to determine the negative of a vector given in 1D and 2D (example questions 15-18 in the supplementary material). This way students came across the concept of understanding the graphical representation of negative of a vector.
Several questions were added (example questions 11, 12, and 14 in the supplementary material) where students had to subtract the vectors. Several variations of these questions are mentioned in the supplementary material. Same questions were asked in a variety of ways, for example, C-D was also asked as −D+C, or D-C was also asked as −C+D. PAL reading content was modified to emphasize the steps to be taken for vector subtraction, such as, students were instructed to first find the negative of a vector to be subtracted and then use head-to-tail method of vector addition. 5.2.5. Graphical scalar multiplication (question 6). Table 9 presents students' performance on scalar multiplication, question 6. Vector A is drawn with its tail at the origin of the coordinate system and represents −2i+2j, and the question asks for graphic representation of −3A. Answer to this question was graphical representation of 6i-6j with the vector's tailat Table 7. Students' answer on question 11. The correct answer is option E. X 2 (1, N = 132) = 35.129, p < .01.

Answer choice
Answer choices (Graphically represented)

Control group
Experimental group the origin of the coordinate system. To answer this question, students need to know the meaning of negative of a vector in addition to relating the change in the magnitude of the vector to the length of the arrow representing the vector. For the experimental group, 79% students could correctly answer this question while only 19% students in control group answered this question correctly. The most common error in the control and experimental group was students choosing the option where the vector 3i-3j was represented graphically with its tail at the origin of the coordinate system. 33% and 10% students in the control group and experimental group, respectively, chose this option.
Looking at students' difficulty in the control group, to facilitate students improve their graphical as well as analytical skills for understanding vector multiplication, variety of questions were included (example Questions 15-20 in the supplementary material) in the PAL question bank where students could (a) draw the vector using widgets when the vector was multiplied by a negative or positive scalar, (b) graphically determine the negative of a vector, and (c) calculate the numerical value of the magnitude of the vector and determine the direction when the vector was multiplied by a negative or positive scalar. Additionally, there were several problems where students subtracted one vector from the other and for doing this they had to first understand how negative of a vector is drawn. Hence combination of variety of problems helped students improve their performance.

Conclusions
PAL content and its implementation strategies where students could practice variety of problems several times throughout the semester resulted in students' correct responses on several questions in the post-test (questions 2, 3, 4 and 5, in table 2), on average for the two semesters, to range from 70% to 85%. On several other questions (questions, 1, 5, 6, 7, 10, 11 and 12, in table 2) students' performance in the post-test for the experimental group improved by about 20% or more compared to the control group. This article presents several questions used in the PAL assignments in an effort to facilitate students enhance their basic understanding of vectors.
Students' misconceptions and difficulties in vectors were addressed by research-based modifications to PAL which helped students to improve their understanding of several vector concepts. Questions added to PAL question-bank were in different styles and formats to reinforce the concepts. For example, in vector addition questions where the students had to determine the magnitude and direction of the resultant vector, the vectors to be added were presented in unit-vector notation, graphically, explained in words. Table 9. Students' response to question 6. In answer choices A, B, C, and E, the tail of the vector is at the origin. In answer choice D the tail of the vector is at (0, −3). X 2 (1, N = 132) = 46.186, p < .01.

Answer choice
Answer choices (graphically represented)

Control group
Experimental group For all types of questions, such as vector addition, subtraction and multiplication, questions were added where students could practice and improve, both, mathematical as well as graphical skills since understanding of one complements the other. A common issue in College Physics 1 course is, students often find it easy to determine the direction of a vector using inverse trigonometric function but mostly make errors when the direction is asked relative to some other axis and not relative to the axis with which they have determined the angle. We addressed this issue by having multiple-choice options where the vector direction was represented relative to different axes and in various ways, such as, above or below the +x or −x axis, relative to +y or −y axis, clockwise or counter-clockwise from certain axis. Also, to address this issue, in several questions itself the directions were given in different styles, such as counter-clockwise or clockwise from a certain axis, or east of north or north of east, or above or below +x or −x axis, so that to solve such problems, students had to first understand the way vector direction was presented. Hence students learn not just by answering the questions but also by reading a variety of questions and learning different ways a vector can be presented in the question.
PAL helped students in working and reviewing the questions as many times as they wanted and promoted self-learning. This is consistent with Arons' [19] suggestion that 'more exercises and drills in graphical handling of vector arithmetic' would benefit students. PAL encouraged the students to redo the assignments without worrying about the risk of lowering the grades since, from multiple attempts, highest score was counted towards the overall grade in the course. PAL feature, such as students flagging the questions helped the instructor to facilitate students' learning and addressing the issues on individual basis or as a class when same question was flagged by several students. Implementation of PAL along with effective research-based modifications to the PAL question bank helped students improve their performance on several vector concepts.
One limitation of this research is that the study is only from one institution. In future studies, we would like to extend this work to other institutions and analyse the effect of PAL on student learning. We plan to extend this work to study student understanding of two important topics, projectile motion, and applications of Newton's laws, to problem-solving.

Acknowledgment
This work was supported by the Course Redesign Initiative grant from the Center for Distributed Learning (CDL) at the University of Central Florida. We thank Mr Joseph Lloyd from CDL for entering several questions in the Realizeit platform.

Data availability statement
Due to university regulation, the data that support the findings of this study are available upon reasonable request from the authors.