Safety Outlier Explorer: test evidence

SOE-FUNC-001 This drop-down menu is used to filter the lab variable that is displayed in the chart. The default measure is set to the first variable in the data set, but the default value can be adjusted to any measure upon the project team's request. Users can select any pre-defined measure in this filter. Selecting a measure will display the data points associated with that measure and will display the visits that the measure is collected at underneath the chart.

SOE-FUNC-002 The chart can be configured to have any number of data filters. Each specified filter has a drop-down menu that is used to filter the visit data that is displayed in the chart.

SOE-FUNC-004 The chart contains a toggle that controls the display of the x-axis. Users can view the x-axis data by study day, visit, or visit number. The x-axis displays the data point at which the measure data is collected and manipulating it changes the scale at which the measure data is plotted. Study day, visit, or visit number can be chosen for a given measure, but the chart default is to show the x-axis by study day.

SOE-FUNC-005 Users are able to adjust the lower limit of the y-axis by manipulating the input within the Lower Limit box. Users are able to adjust the upper limit of the y-axis by manipulating the input within the Upper Limit box. Each limit box has up and down arrows to adjust the value of the box by 1. Users can select an arrow to change the input or delete the current value in the box and type in a new value. When typing in a value, users must click outside of the box to apply the changes. When a user manipulates a y-axis limit, the y-axis on the chart below the controls will reflect the change.

SOE-FUNC-006 Users can change the upper and lower y-axis limits back to the default values for a measure by selecting the "Reset Limits" button. Pressing the button will erase the current values in the upper and lower limit boxes and replace them with defaults. The chart below will also reflect the changes and will show the default axes.

SOE-FUNC-007 Users can opt to change the method for the normal ranges to the following options: None, LLN-ULN, Standard Deviation, and Quantiles. The default for the normal range controls is set to "LLN-ULN", which is the upper and lower normal range specified in the data. Users may select "None", which will remove the green shading around the normal ranges and will remove the inlier count to the left of the charts. Users may also select "Standard Deviation", which will prompt the user to enter the number of standard deviations they would like to set as the normal range. Altering the default value will change the upper and lower normal range bands in the chart and will change the inlier count to the left of the chart. Users may also select "Quantiles", which prompts the user to enter an upper and lower quantile to set as the normal ranges. Altering the upper quantile will change the upper normal range band in the chart and altering the lower quantile will change the lower normal range band in the chart. Changing the normal ranges may change the number of inliers and the inlier count to the left of the chart will update based on your selections.

SOE-FUNC-003 Displayed under the header, this statement provides the total number of participants in the data set in integers as well as the percentage of participants shown to one decimal place. In the default view, all participant data is included in the chart, or 100%, unless the project team requests otherwise. When filters are used, the data shown in the histogram is manipulated and the number of participant records shown at the top of the page may increase or decrease, depending on the variable selected.

SOE-REG-001 Confirm that the number and percentage of participants is captured above the chart

SOE-REG-002 Confirm that selecting the Measure filter changes the distribution of observations

SOE-REG-037 Confirm that the right side of the webpage between the controls and the chart a note should appear that details how many records with missing and/or non-numeric results were removed.

SOE-REG-038 Confirm that the test dataset already contains missing records. To test non-numeric records edit ADBDS.csv by adding non-numeric values to the STRESN column.

SOE-FUNC-010 Clicking a data point will open a listing of small multiples below the chart that contains a detailed listing of all of the participant's records. This linked table contains small charts for each measure captured in the data set as well as the participant's subject ID, age, sex, and race. When the listing appears, the data points selected will appear in a continuous gray line inside of the chart. When a user clicks outside of the selected data point, the gray line disappears along with the detailed listing.

SOE-FUNC-012 Display the raw data records, including the subject identifier, age, race, sex, and small data renderings of each measure captured in the data set

SOE-FUNC-013 Create a gray line to connect the selected data points

SOE-REG-013 Confirm that clicking on a data point applies a dark highlight to all of the participant's data points

SOE-REG-014 Confirm that clicking on a data point opens a detailed listing of a participant's data beneath the chart

SOE-REG-016 Confirm that all marks (points, lines) associated with that participant should highlight.

SOE-REG-022 Confirm that the listing contains subject-level demographic data (Age, Sex, and Race appear by default, can add other Participant Details if wanted)

SOE-REG-023 Confirm that the listing contains subject-level information about measures and and visits

SOE-FUNC-010 Clicking a data point will open a listing of small multiples below the chart that contains a detailed listing of all of the participant's records. This linked table contains small charts for each measure captured in the data set as well as the participant's subject ID, age, sex, and race. When the listing appears, the data points selected will appear in a continuous gray line inside of the chart. When a user clicks outside of the selected data point, the gray line disappears along with the detailed listing.

SOE-REG-020 Confirm that with an overlapping point highlighted, changing any control should remove the note but maintain the highlighting of the selected participant.

SOE-FUNC-007 Users can opt to change the method for the normal ranges to the following options: None, LLN-ULN, Standard Deviation, and Quantiles. The default for the normal range controls is set to "LLN-ULN", which is the upper and lower normal range specified in the data. Users may select "None", which will remove the green shading around the normal ranges and will remove the inlier count to the left of the charts. Users may also select "Standard Deviation", which will prompt the user to enter the number of standard deviations they would like to set as the normal range. Altering the default value will change the upper and lower normal range bands in the chart and will change the inlier count to the left of the chart. Users may also select "Quantiles", which prompts the user to enter an upper and lower quantile to set as the normal ranges. Altering the upper quantile will change the upper normal range band in the chart and altering the lower quantile will change the lower normal range band in the chart. Changing the normal ranges may change the number of inliers and the inlier count to the left of the chart will update based on your selections.

SOE-REG-025 Confirm that a dropdown labeled Normal range method appears in the controls section with four options: None, LLN-ULN, Standard Deviation, and Quantiles.

SOE-REG-026 Confirm that selecting Standard Deviation displays a text input labeled Number of standard deviations and defaults to 1.96.

SOE-REG-027 Confirm that selecting Quantiles displays two text inputs, one for the lower quantile and one for the upper quantile, labeled Lower quantile and Upper quantile respectively, that default to .05 and .95.

SOE-FUNC-005 Users are able to adjust the lower limit of the y-axis by manipulating the input within the Lower Limit box. Users are able to adjust the upper limit of the y-axis by manipulating the input within the Upper Limit box. Each limit box has up and down arrows to adjust the value of the box by 1. Users can select an arrow to change the input or delete the current value in the box and type in a new value. When typing in a value, users must click outside of the box to apply the changes. When a user manipulates a y-axis limit, the y-axis on the chart below the controls will reflect the change.

SOE-FUNC-006 Users can change the upper and lower y-axis limits back to the default values for a measure by selecting the "Reset Limits" button. Pressing the button will erase the current values in the upper and lower limit boxes and replace them with defaults. The chart below will also reflect the changes and will show the default axes.

SOE-REG-004 Confirm that entering a value into the Y-axis Lower Limit box and clicking outside of the box updates the lower limit of the y-axis

SOE-REG-005 Confirm that entering a value into the Upper Limit box and clicking outside of the box updates the upper limit of the y-axis

SOE-REG-006 Confirm that selecting the y-axis reset button returns the chart to the default values for that measure

SOE-REG-048 Add the following snippet: groups: [{value_col: 'ARM',label: 'Treatment Group'},{value_col: 'SEX',label: 'Sex'}] and verify that the chart now has a "Group By" control.

SOE-REG-049 Confirm that the points and lines are colored by the categories of that group, and that a legend appears at the bottom corresponding to the current grouping and colors.

SOE-REG-050 Confirm that when the 'Group by' Control is changed, the chart and legend reflects the new grouping.

SOE-FUNC-004 The chart contains a toggle that controls the display of the x-axis. Users can view the x-axis data by study day, visit, or visit number. The x-axis displays the data point at which the measure data is collected and manipulating it changes the scale at which the measure data is plotted. Study day, visit, or visit number can be chosen for a given measure, but the chart default is to show the x-axis by study day.

SOE-REG-003 Confirm that the x-axis filter changes the x-axis to the selected variable

SOE-REG-011 Confirm that hovering on a data point displays tooltips for the data point with Subject, value, and visit

SOE-API-003 The custom participantsSelected event is dispatched to the overall chart wrapper (chart.wrap) whenever the details for a given participant are viewed (or cleared) by clicking on a line or point in the chart. The event has a custom data property holding an array with the selected ID when a new participant is selected (["123-456-7"]) or an empty array when participant details are cleared.

SOE-REG-051 Add the following snippet: "filters": [{"value_col": "ARM","label": "Treatment Group","start": "Placebo"}] and verify a filter appears with the name 'Treatment Group' that subsets the data appropriately when used.

SOE-REG-052 Verify that the dropdown for the the filter does not have an "All" option

SOE-REG-053 Verify that on renderering, the chart is already filtered by 'Placebo' and that this is reflected by the drop down

SOE-API-001 a factory to create a custom Webcharts chart object

SOE-AXIS-001 On load and after every render, each Y-axis Limits input displays the limit currently in force for the selected measure — the padded default domain of the filtered results (SOE-REG-034) by default, or the user's override — formatted to three significant figures of the axis range. Neither box is ever blank.

SOE-AXIS-002 Displaying a limit does not pin it. A limit the user has not edited remains automatic — it is re-derived from the data on every render, so changing the measure, the filters, or the y-axis scale updates the value shown — while an edited limit persists as an override until it is reset. The distinction is held in state (an override slot separate from the recorded domain), not inferred from an empty input.

SOE-AXIS-003 The Reset Limits control clears both y-axis limits overrides and repopulates both inputs with the derived limits, so there is always a way back to automatic now that an empty box no longer means it.

SOE-CFG-004 an array of filter variables and associated metadata

SOE-CFG-005 an array of ID-level variables and associated metadata

SOE-CFG-006 added in v2.4. Takes an array of col/label objects and adds those values to the end of the standard point tooltips. The following would add Date to the tooltip: {tooltip_cols: [{label:"Date",value_col:"DT"}]};

SOE-CFG-013 an object that defines the line color, thickness, and opacity

SOE-CFG-014 an object that defines the point color, radius, and opacity and its outline color, thickness, and opacity

SOE-CFG-007 the number of standard deviations from the mean with which to define the normal range

SOE-CFG-008 this quantile defines the lower bound of the normal range

SOE-CFG-009 this quantile defines the upper bound of the normal range

SOE-FUNC-004 The chart contains a toggle that controls the display of the x-axis. Users can view the x-axis data by study day, visit, or visit number. The x-axis displays the data point at which the measure data is collected and manipulating it changes the scale at which the measure data is plotted. Study day, visit, or visit number can be chosen for a given measure, but the chart default is to show the x-axis by study day.

SOE-REG-048 Add the following snippet: groups: [{value_col: 'ARM',label: 'Treatment Group'},{value_col: 'SEX',label: 'Sex'}] and verify that the chart now has a "Group By" control.

SOE-REG-037 Confirm that the right side of the webpage between the controls and the chart a note should appear that details how many records with missing and/or non-numeric results were removed.

SOE-REG-038 Confirm that the test dataset already contains missing records. To test non-numeric records edit ADBDS.csv by adding non-numeric values to the STRESN column.

SOE-REG-029 Confirm that a unit variable is appended to each measure in the Measure dropdown.

SOE-REG-031 Confirm that there are values displayed next to each measure under the specified unit variable in the data set.

SOE-FUNC-004 The chart contains a toggle that controls the display of the x-axis. Users can view the x-axis data by study day, visit, or visit number. The x-axis displays the data point at which the measure data is collected and manipulating it changes the scale at which the measure data is plotted. Study day, visit, or visit number can be chosen for a given measure, but the chart default is to show the x-axis by study day.

SOE-REG-028 Confirm that the x-axis tick labels change rotation angles depending on the x-axis control variable.

SOE-FUNC-007 Users can opt to change the method for the normal ranges to the following options: None, LLN-ULN, Standard Deviation, and Quantiles. The default for the normal range controls is set to "LLN-ULN", which is the upper and lower normal range specified in the data. Users may select "None", which will remove the green shading around the normal ranges and will remove the inlier count to the left of the charts. Users may also select "Standard Deviation", which will prompt the user to enter the number of standard deviations they would like to set as the normal range. Altering the default value will change the upper and lower normal range bands in the chart and will change the inlier count to the left of the chart. Users may also select "Quantiles", which prompts the user to enter an upper and lower quantile to set as the normal ranges. Altering the upper quantile will change the upper normal range band in the chart and altering the lower quantile will change the lower normal range band in the chart. Changing the normal ranges may change the number of inliers and the inlier count to the left of the chart will update based on your selections.

SOE-CFG-007 the number of standard deviations from the mean with which to define the normal range

SOE-CFG-008 this quantile defines the lower bound of the normal range

SOE-REG-025 Confirm that a dropdown labeled Normal range method appears in the controls section with four options: None, LLN-ULN, Standard Deviation, and Quantiles.

SOE-FUNC-005 Users are able to adjust the lower limit of the y-axis by manipulating the input within the Lower Limit box. Users are able to adjust the upper limit of the y-axis by manipulating the input within the Upper Limit box. Each limit box has up and down arrows to adjust the value of the box by 1. Users can select an arrow to change the input or delete the current value in the box and type in a new value. When typing in a value, users must click outside of the box to apply the changes. When a user manipulates a y-axis limit, the y-axis on the chart below the controls will reflect the change.

SOE-FUNC-006 Users can change the upper and lower y-axis limits back to the default values for a measure by selecting the "Reset Limits" button. Pressing the button will erase the current values in the upper and lower limit boxes and replace them with defaults. The chart below will also reflect the changes and will show the default axes.

SOE-REG-004 Confirm that entering a value into the Y-axis Lower Limit box and clicking outside of the box updates the lower limit of the y-axis

SOE-REG-033 Confirm that when clicking the arrows in the y-axis limit inputs, the limits should increment by a reasonable factor of the y-axis range, or roughly 1/15 of the range. For example, if the range is 150 the increment should be 10 and if the range is 1.5 the increment should be .1.

SOE-REG-028 Confirm that the x-axis tick labels change rotation angles depending on the x-axis control variable.

SOE-REG-011 Confirm that hovering on a data point displays tooltips for the data point with Subject, value, and visit

SOE-REG-049 Confirm that the points and lines are colored by the categories of that group, and that a legend appears at the bottom corresponding to the current grouping and colors.

SOE-CFG-006 added in v2.4. Takes an array of col/label objects and adds those values to the end of the standard point tooltips. The following would add Date to the tooltip: {tooltip_cols: [{label:"Date",value_col:"DT"}]};

SOE-DATA-001 The Safety Outlier Explorer is initialized with JSON data files that match the format created by d3.csv(). The chart expects an ADaM-esque data structure by default but can be customized to work with any dataset that contains one row per participant per time point per measure and includes the required columns specified below.

SOE-DATA-003 No customization is needed to initialize a chart when the data object contains all default variables above. Just pass an empty object ({}) for settings and initialize the chart like so: safetyOutlierExplorer('body', {}).init(data).

SOE-API-001 a factory to create a custom Webcharts chart object

SOE-AXIS-001 On load and after every render, each Y-axis Limits input displays the limit currently in force for the selected measure — the padded default domain of the filtered results (SOE-REG-034) by default, or the user's override — formatted to three significant figures of the axis range. Neither box is ever blank.

SOE-AXIS-002 Displaying a limit does not pin it. A limit the user has not edited remains automatic — it is re-derived from the data on every render, so changing the measure, the filters, or the y-axis scale updates the value shown — while an edited limit persists as an override until it is reset. The distinction is held in state (an override slot separate from the recorded domain), not inferred from an empty input.

SOE-AXIS-003 The Reset Limits control clears both y-axis limits overrides and repopulates both inputs with the derived limits, so there is always a way back to automatic now that an empty box no longer means it.

SOE-AXIS-004 Pre-filling must not let an invalid domain reach the chart: a limit typed across the other limit as currently displayed is swapped rather than inverted, an empty or non-numeric entry returns that limit to automatic, and a log-scale domain stays positive.