Skip to content

Windows 10 parallelization can be super slow #94

Closed

Description

@avehtari

Lawrence Hunsicker reported that Windows 10 is super slow in parallelization if .rprofile has options(mc.cores = parallel::detectCores()). The same line in a script is not a problem,
https://discourse.mc-stan.org/t/problem-parallelizing-rstanarm-loo/6433/10

I don't know what we can do for this in loo package, but I add this here if someone is hit by the same Windows bug.

Activity

  1. jgabry commented on Jan 3, 2019

    @jgabry
    Member

    We could mention this in the package documentation, but if this is very severe then maybe it's worth printing a message about it to Windows users when the package is loaded?
    I'm not sure what the message would be precisely, but it's possible by adding to the .onAttach() function. Currently it's this:

    loo/R/zzz.R

    Lines 1 to 10 in e147fd3

    .onAttach <- function(...) {
    ver <- utils::packageVersion("loo")
    packageStartupMessage(
    "This is loo version ", ver, ".\n",
    "**NOTE: As of version 2.0.0 loo defaults to 1 core ",
    "but we recommend using as many as possible. ",
    "Use the 'cores' argument or set options(mc.cores = NUM_CORES) for an entire session. ",
    "Visit mc-stan.org/loo/news for details on other changes."
    )
    }

  2. added a commit that references this issue on Feb 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions