Interactive data science environment runs in a temp directory by default

[gramster]

Environment data

• VS Code version: 1.29.1
• Extension version (available under the Extensions sidebar): 2018.10.1
• OS and version: MacOS 10.14.1
• Python version (& distribution if applicable, e.g. Anaconda): Anaconda, 3.6
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
• Relevant/affected Python packages and their versions: N/A

Actual behavior

If I open a notebook, I expect my working directory to be the notebook directory, so I can load up others files in that directory using paths starting with "./". However, it appears the working directory is a temporary directory (e.g. /private/var/folders/gq/1zp9x1h125j7w4_gbb4kryv00000gn/T)

Expected behavior

Working directory should be notebook directory by default.

Steps to reproduce:

1. Run:

   #%%
   !pwd

[qubitron]

Comment on our blog post, we might consider always defaulting to workspace root, even when the file is saved:

Hello,

great stuff, we’re loving it in our data science team !
However, the server starts in the file folder and not in the opened folder.
Hope it will come soon, it’s frustrating !

But keep up this good work 🙂

[rchiodo]

I think this is going to remain this way for now. We need a notebook file. We generate one in your temp folder. The current directory should still work though. We pass %cd to the kernel before every execute.

[rchiodo]

Hmm. Jupyter doesn't have this problem with a temp notebook though. Maybe there's something we can pass as the current directory.

[rchiodo]

Oh jupyter writes out a temp file to the current folder. That seems really bad for us since the user isn't necessarily generating a notebook

[rchiodo]

Yeah I think we should just fix the cd for a temp python file to be the workspace, everything else should be the file directory (otherwise relative output in that file won't match working python code)

Example:
F:\Foo\SubDir\MyFile.py
pd.read_csv("..\Data.csv")

If the user opens workspace at F:\Foo, then this file doesn't work.

[qubitron]

Rich Chiodo (@rchiodo) the feedback from the user above suggests that they expected to do:

F:\Foo\SubDir\MyFile.py
pd.read_csv("Data.csv")

Which would also be consistent with the behavior of "run python file in terminal", and debugging.

E.g. given a folder structure of:

newcode
  - subdir 
     - myfile.py

With newcode open as the workspace root, and if you F5 on myfile.py or run myfile.py in the terminal, you will see it run out of newcode:

C:\dev\newcode>python c:/dev/newcode/subdir/myfile.py
C:\dev\newcode

C:\dev\newcode>

Myfile.py just prints os.getcwd()

[rchiodo]

So I think what you're saying is that when running a python file, the current directory is always set to the workspace root (and not where the file is)? What about in a multi-root scenario?

[qubitron]

Yes, and also run selection in terminal also uses the workspace root (shift+enter without cells).

For the multi-root scenario, each file belongs to a workspace and so when it is run, it will use the workspace root in which the file belongs.

Adding another folder to the workspace:

newcode
  - subdir 
     - myfile.py
hello5
   - subdir
     - anotherfile.py

Running anotherfile.py:

C:\dev\hello5>python c:/dev/hello5/subdir/anotherfile.py
C:\dev\hello5

[rchiodo]

Okay that should be an easy fix. Instead of cd to the file directory at the beginning of the execute, we cd to the workspace root of the file, or the first root if the file doesn't exist.

Thanks Dan Taylor (@qubitron)

[ronglums]

unless the file is saved to a location that's outside of the workspace.

[qubitron]

Rong Lu (@ronglums) even for this case we run the file in whatever the current workspace root is, e.g. I opened this file outside of the current workspace:

C:\dev\hello5>python c:/dev/hello3/hello.py
Hello world!

Not saying it's necessarily the correct behavior, but in absent of strong feedback being consistent is a good default choice.

[gramster]

Let’s be consistent with what Jupyter does (I think that is based on file location but I’m not sure). That way an imported noteboook should just work. BTW made it to LAX. Will get to Auckland eventually.

…

On Tue, Nov 20, 2018 at 5:34 PM Dan Taylor ***@***.***> wrote: @ronglums <https://github.com/ronglums> even for this case we run the file in whatever the current workspace root is, e.g. I opened this file outside of the current workspace: C:\dev\hello5>python c:/dev/hello3/hello.py Hello world! Not saying it's necessarily the correct behavior, but in absent of strong feedback being consistent is a good default choice. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/Microsoft/vscode-python/issues/3407#issuecomment-440495506>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AConiDZB25OnKVxSiYLxZa6isEcqGMs3ks5uxK2ggaJpZM4YqIEW> .

[qubitron]

Graham Wheeler (@gramster) good point, I checked and Jupyter does seem to use the file location (not where you run jupyter notebook out of).

[benjnr]

Comment on our blog post, we might consider always defaulting to workspace root, even when the file is saved:

Hello,

great stuff, we’re loving it in our data science team !
However, the server starts in the file folder and not in the opened folder.
Hope it will come soon, it’s frustrating !

But keep up this good work 🙂

Hello there, that's my post on the blog !
yeah, I wasn't clear.

The question is indeed the mismatch between jupyter behaviour (workdir is file dir) and python (workdir is folder dir), which is inconsistent.

Dan Taylor (@qubitron) pointed the python behaviour. I think it's the logic one as it would not hinder interactive code and promote good py packaging practice.

Maybe a feature request (as a user setting) more than a bug ?