7.4. Working with Problem Components¶

7.4.4.1. The Simple Editor¶

When you select the following problem types, the simple editor opens with a preformatted example problem.

• Checkbox Problem: In checkbox problems, learners select one or more options from a list of possible answers.
• Dropdown Problem: In dropdown problems, learners select one answer from a dropdown list.
• Multiple Choice Problem: Multiple choice problems require learners to select one answer from a list of choices that appear below the problem text.
• Numerical Input Problem: Numerical input problems require answers that include only integers, fractions, and a few common constants and operators.
• Text Input Problem: In text input problems, learners enter a short text answer.

The following image shows an example multiple choice problem in the simple editor.

The simple editor includes a toolbar with options that provide the required formatting for different types of problems. When you select an option from the toolbar, formatted sample text appears in the simple editor. Alternatively, you can apply formatting to your own text by selecting the text and then one of the toolbar options.

Descriptions of the toolbar options follow.

1. Heading: Formats text as a title or heading.
2. Multiple Choice: Identifies text as an answer option for a multiple choice problem.
3. Checkboxes: Identifies text as an answer option for a checkboxes problem.
4. Text Input: Identifies text as the correct answer for a text input problem.
5. Numerical Input: Identifies the correct answer, with an optional tolerance, for a numerical input problem.
6. Dropdown: Identifies a comma-separated list as correct and incorrect answer options for a dropdown problem.
7. Explanation: Formats text as an explanation that appears after learners select Show Answer.
8. Opens the problem in the advanced editor.
9. Opens a list of formatting hints.
10. Accessible Label: Identifies the part of the problem text that is the specific question that learners will answer by selecting the options that follow, or by entering a text or numeric response. The toolbar does not have an option that provides this formatting, so you type two angle brackets on either side of the question text pointing inward. For example, >>Is this the question text?<<.
    • Screen readers read all of the text that you supply for the problem, and then repeat the text that you identify with this formatting immediately before reading the answer choices for the problem.
    • The Student Answer Distribution report uses the text with this formatting to identify each problem.
    • Insights also uses the text with this formatting to identify each problem. For more information, see Using edX Insights.

7.4.4.2. The Advanced Editor¶

The advanced editor opens a problem in XML. Templates for problems such as such as drag and drop and math expression input open directly in the advanced editor.

The following image shows the multiple choice problem above in the advanced editor instead of the simple editor.

The following problem templates open in the advanced editor.

• Circuit Schematic Builder Problem In circuit schematic problems, learners create and modify circuits on an interactive grid and submit computer- generated analyses of the circuits for grading.
• Custom JavaScript Problem With custom JavaScript problems, you can create a custom problem or tool that uses JavaScript and add it directly into Studio.
• Drag and Drop Problem (Deprecated) Drag and drop problems require learners to drag text or objects to a specific location on an image.
• Image Mapped Input Problem Image mapped input problems require learners to select a specific location on an image.
• Math Expression Input Problems Math expression input problems require learners to enter a mathematical expression as text, such as e=m*c^2.
• Problem with Adaptive Hint These problems can give learners feedback or hints based on their responses. Problems with adaptive hints can be text input or multiple choice problems.
• Problem Written in LaTeX This problem type allows you to convert problems that you previously wrote in LaTeX into the edX format. Note that this problem type is a prototype, and is not supported.
• Write-Your-Own-Grader Problem Custom Python-evaluated input (also called “write-your-own-grader” problems evaluate learners’ responses using an embedded Python script that you create. These problems can be of any type.

7.4.5.1. Display Name¶

This setting indicates the name of your problem. This name appears as a heading above the problem in the LMS, and it identifies the problem for you in Insights.

The following illustration shows the display name of a problem in Studio, in the LMS, and in Insights.

Each problem type supplies a default display name that identifies the type of problem component added. Changing the default to a unique, descriptive display name can help you and your learners identify different problems quickly and accurately. If you delete the default display name and do not enter your own identifying name, the platform supplies “problem” for you.

For more information about metrics for your course’s problem components, see Using edX Insights.

7.4.5.2. Maximum Attempts¶

This setting specifies the number of times that a learner is allowed to attempt answering a problem. By default, the course-wide Maximum Attempts advanced setting is null, meaning that the maximum number of attempts for problems is unlimited. If the course-wide Maximum Attempts setting is changed to a specific number, the Maximum Attempts setting for individual problems defaults to that number, and cannot be set to unlimited.

7.4.5.3. Problem Weight¶

This setting specifies the total number of points possible for the problem. The problem weight appears next to the problem’s display name.

By default, each response field, or “answer space”, in a problem component is worth one point. Any problem component can have multiple response fields. For example, the problem component above contains one dropdown problem that has three separate questions, and also has three response fields.

You can increase or decrease the weight for a problem component by entering a Problem Weight.

7.4.5.4. Randomization¶

For problems that include a Python script to generate numbers randomly, this setting specifies how frequently the values in the problem change: each time a different learner accesses the problem, each time a single learner tries to answer the problem, both, or never.

For example, in this problem, the highlighted values change each time a learner submits an answer to the problem.

If you want to randomize numeric values in a problem, you complete both of these steps.

• Make sure that you edit your problem to include a Python script that randomly generates numbers.
• Select an option other than Never for the Randomization setting.

The edX Platform has a 20-seed maximum for randomization. This means that learners see up to 20 different problem variants for every problem that has Randomization set to an option other than Never. It also means that every answer for the 20 different variants is reported by the Answer Distribution report and edX Insights. Limiting the number of variants to a maximum of 20 allows for better analysis of learner submissions by allowing you to detect common incorrect answers and usage patterns for such answers.

For more information, see Student Answer Distribution in this guide, or Review Answers to Graded Problems or Review Answers to Ungraded Problems in Using edX Insights.

You can choose the following options for the Randomization setting.

7.4.5.5. Show Answer¶

This setting defines when learners are shown the answers to a problem and has the following options

7.4.5.6. Show Reset Button¶

This setting defines whether a Reset button is visible on the problem.

Learners can select Reset to clear any input that has not yet been submitted, and try again to answer the problem.

If the learner has already submitted an answer, selecting Reset clears the submission and, if the problem contains randomized variables and randomization is set to On Reset, changes the values in the problem.

If the number of Maximum Attempts that was set for this problem has been reached, the Reset button is not visible.

This problem-level setting overrides the course-level Show Reset Button for Problems setting.

7.4.5.7. Timer Between Attempts¶

This setting specifies the number of seconds that a learner must wait between submissions for a problem that allows multiple attempts. If the value is 0, the learner can attempt the problem again immediately after an incorrect attempt.

Adding required wait time between attempts can help to prevent learners from simply guessing when multiple attempts are allowed.

If a learner attempts a problem again before the required time has elapsed, she sees a message below the problem indicating the remaining wait time. The format of the message is, “You must wait at least {n} seconds between submissions. {n} seconds remaining.”

7.4.6.1. Workarounds¶

If you have to modify a released problem in a way that affects grading, you have two options to ensure that every learner has the opportunity to submit a new response and be regraded. Note that both options require you to ask your learners to go back and resubmit answers to a problem.

• In the problem component that you changed, increase the number of attempts for the problem, then ask all your learners to redo the problem.
• Delete the entire problem component in Studio and create a new Problem component with the content and settings that you want, then ask all your learners to complete the new problem. (If the revisions you must make are minor, duplicate the problem component before you delete it and revise the copy.)

For information about how to review and adjust learner grades in the LMS, see Learner Grades and Grading.

7.4.8.1. Feedback in Response to Attempted Answers¶

You can add feedback that is displayed to learners when they submit an answer.

For example, the following multiple choice problem provides feedback in response to the selected option when the learner selects Check. In this case, feedback is given for an incorrect answer.

7.4.8.2. Best Practices for Providing Feedback¶

The immediacy of the feedback available to learners is a key advantage of online instruction and difficult to do in a traditional classroom environment.

You can target feedback for common incorrect answers to the misconceptions that are common for the level of the learner (for example, elementary, middle, high school, college).

In addition, you can create feedback that provides some guidance to the learner about how to arrive at the correct answer. This is especially important in text input and numeric input problems, because without such guidance, learners might not be able to proceed.

You should also include feedback for the correct answer to reinforce why the answer is correct. Especially in questions where learners are able to guess, such as multiple choice and dropdown problems, the feedback should provide a reason why the selection is correct.

7.4.8.3. General Hints to a Problem¶

You can add one or more hints that are displayed to learners. When you add hints, the Hint button is automatically displayed to learners. Learners can access the hints by selecting Hint beneath the problem. A learner can view multiple hints by selecting Hint multiple times.

For example, in the following multiple choice problem, the learner selects Hint before attempting the answer.

The hint indicates that it is the first of two hints. The learner selects Hint a second time.

7.4.8.4. Best Practices for Providing Hints¶

To ensure that your hints can assist learners with varying backgrounds and levels of understanding, you should provide multiple hints with different levels of detail.

For example, the first hint can orient the learner to the problem and help those struggling to better understand what is being asked.

The second hint can then take the learner further towards the answer.

In problems that are not graded, the third and final hint can explain the solution for learners who are still confused.

7.4.8.5. Create Problems with Feedback and Hints¶

You create problems with feedback and hints in Studio. Templates with feedback and hints configured are available to make creating your own problems easier.

While editing a unit, in the Add New Component panel, select Problem. In the list that opens, select the Common Problem Types tab. Templates for problems with feedback and hints are listed.

Add the problem type you need to the unit, then edit the component. The exact syntax you use to configure hints and feedback depends on the problem type. See the topic for the problem type for more information.

• Checkbox Problem
• Dropdown Problem
• Multiple Choice Problem
• Numerical Input Problem
• Text Input Problem

7.4.9.1. How Learners Receive Partial Credit¶

Learners receive partial credit when they submit an answer in the LMS.

In the following example, the course team configured a multiple choice problem to award 25% of the possible points (instead of 0 points) for one of the incorrect answer options. The learner selected this incorrect option, and received 25% of the possible points.

7.4.9.2. Partial Credit and Reporting on Learner Performance¶

When a learner receives partial credit for a problem, the LMS only adds the points that the learner earned to the grade. However, the LMS reports any problem for which a learner receives credit, in full or in part, as correct in the following ways.

• Events that the system generates when learners receive partial credit for a problem indicate that the answer was correct. Specifically, the correctness field has a value of correct.

  For more information about events, see Problem Interaction Events in the EdX Research Guide.

• The AnswerValue in the Student Answer Distribution report is 1, for correct.

• The edX Insights insights:student performance reports count the answer as correct.

Course teams can see that a learner received partial credit for a problem in the learner’s submission history. The submission history shows the score that the learner received out of the total available score, and the value in the correctness field is partially-correct. For more information, see Learner Answer Submissions.

7.4.10.1. Course Outline Terminology in Exported Files¶

Sections, subsections, units, and components have different names in the Course Outline view and in the list of files that you will see after you export your course and open the .xml files for editing. The following table lists the names of these elements in the Course Outline view and in a list of files.

For example, when you want to find a specific section in your course, look in the Chapter directory when you open the list of files that your course contains. To find a unit, look in the Vertical directory.

7.4.10.2. Create Randomized Problems¶

You can provide different learners with different problems by using randomized content blocks, which randomly draw problems from pools of problems stored in content libraries. For more information, see Randomized Content Blocks.

7.4.10.3. Adding Tooltips to a Problem¶

To help learners understand terminology or other aspects of a problem, you can add inline tooltips. Tooltips show text to learners when they move their cursors over a tooltip icon.

The following example problem includes two tooltips. The tooltip that provides a definition for “ROI” is being shown.

To add the tooltip, you wrap the text that you want to appear as the tooltip in the clarification element. For example, the following problem contains two tooltips.

<problem>
    <text>
        <p>Given the data in Table 7 <clarification>Table 7: "Example PV
          Installation Costs", Page 171 of Roberts textbook</clarification>,
          compute the ROI <clarification><strong>ROI</strong>: Return on
          Investment</clarification> over 20 years.
        </p>
     . . .