July 10th, 2019

Expert Elicitation Workflow

  • Toxic effects of spray for killing mosquitos
  • Personnel selection
  • Temperature in Toulouse next Wednesday

Specific Features

Visualizations

  • Draggable barplot with plotly
  • Combined distribution of all experts’ elicitations

Automation

  • Send e-mails using mailR
  • Enter quantiles in specific order using rhandsontable

Deployment

  • Save results in single folder per session with docker volume
  • Authorization via ShinyProxy

Live Demo

Send e-mails

mailR::send.mail(
        from = "no-reply <noreply-efsa@openanalytics.eu>",
        to = emails,
        subject = input$subject, 
        body = input$message,
        smtp = list(
                host.name = "smtp.openanalytics.eu", 
                port = 25, 
                user.name = "noreply-efsa", 
                passwd = "***", 
                ssl = TRUE
        ),
        authenticate = TRUE, 
        attach.files = 
                if (is.null(dossier))
                    NULL else
                    file.path(workingDir, dossier)
)

Enter Quantiles

library(rhandsontable)
rhandsontable(data, readOnly = TRUE) %>%
        hot_col(col = 4, type = "numeric") %>%
        hot_cell(row = editRow() + 1, col = 4, readOnly = FALSE) %>%
        # To make sure enough digits are shown
        hot_cols(renderer = htmlwidgets::JS("safeHtmlRenderer")) %>%
        hot_cols(renderer = paste0("
                                function (instance, td, row, col) {
                                Handsontable.renderers.TextRenderer.apply(this, arguments);
                                var row_index = ", paste("[", paste(editRow(), collapse = ","), "]"), ";
                                for (i = 0; i < row_index.length; i++)
                                if (row == row_index[i] & col == 3) {
                                td.style.background = '", editColor(), "';
                                }
                                }"))

Draggable Barplot

library(plotly)
plot_ly(data = plotData, 
                x = ~x, y = ~c(0, diff(y)/diff(x)),
                type = "scatter", mode = "lines",
                line = list(shape = "vh"), fill = "tozeroy") %>%
        # add vertical lines
        layout(shapes = lapply(plotData$x, function(xx)
                            list(type = "line", x0 = xx, x1 = xx, y0 = 0, y1 = 1, yref = "paper"))) %>%
        # allow to drag lines
        config(edits = list(shapePosition = TRUE)) 

Combined Distribution

↓

↓

Best fitted distribution
to suggest agreed quantile values

↓

0.001 0.25 0.5 0.75 0.999
28 32 35 36 38

Authorization

With ShinyProxy
Username is passed from LDAP login to R Shiny App

login <- Sys.getenv("SHINYPROXY_USERNAME")



Facilitator Expert
Who list with limited no. of people (extdata folder) per session defined by the facilitator
Action create/close session, send invites, report results elicitation
Check login in admin list login in session’s experts list

Save Results

Create Docker Volume

  • Single folder per session
  • Not single rectangular data file
  • Persistent data storage (remote)

Session

Administrative

  • Experts (.csv)
  • Parameters to be Elicited (.csv)
  • Cutpoints to Judge (.csv)
  • Evidence dossier (.pdf, .docx)

Logging Elicitation

  • Range (.csv)
  • Quantiles (.csv)

Thank You

Screenshot: Manage Session

Screenshot: Draggable Barplot

Screenshot: Combined Distributions

Screenshot: Folder Structure (1/2)

Screenshot: Folder Structure (2/2)

Screenshot: Define Cutpoints