Troubleshooting RStudio IDE Connection Errors

Overview

Use this guide to troubleshoot connection errors for RStudio Desktop, Server, and Workbench. Leave a comment if you don't find a solution to your problem.

Errors starting a new session

Errors like these are displayed when failing to connect to a new session. The dialog's title is one of: RStudio Initialization Error, or Cannot Connect to R Session. Look in the session output to see if the session failed to start, is slow starting and what port it's listening on. Check if the R session is still running or has crashed.

Errors when using the IDE

These errors may show up in the R console that an R operation has failed (e.g. console_input), or as dialog boxes that indicate a particular UI operation has failed (e.g. Save File).

Some errors show the specific HTTP status code that caused the failure:

If the error persists, most likely the R session is no longer running. Look in the session output or check the session status to determine if it's still running or has crashed with an exit error message. If the R session is still running and using RStudio Server, look in rserver.log for a message that correlates with the error. If the session exited unexpectedly see this section. It not, it could be a load balancing error or networking error.

Errors in the homepage

When using RStudio Workbench, a failed session start or crashed session shows a status of Suspended or Killed on the homepage:

In this case, look in the session output, rserver.log and see if the session exited unexpectedly.

If the homepage itself fails to load, it could be that the rworkspaces program failed to start for that user. It requires PAM session access on the RSW node and access to the user's home directory and project directories. Look for rworkspaces-username.log in the ~/.local/share/rstudio/log directory to see if any errors have occurred. Also look in rserver.log.

Known issues

If your problem occurs while R is busy, you may benefit from an upgrade if your RStudio version is earlier than 2021.09.0. That version introduced Async RPC, a feature that improves connection management in the IDE in this situation.

If you see the R error recursive gc invocation and are on 2021.09.0 upgrade to 2022.02.0 or later release or set session-handle-offline-enabled=0 in /etc/rstudio/rsession.conf.

Is the R session running?

Unless the R console or terminal responds after the error, it might be worth checking if the rsession process is running, failed to start or crashed. You can usually tell from the session output if it's still running, what port it is listening to and if it crashed with an error. If not you can confirm the status in a few ways:

For RStudio Workbench:

• Use the homepage to find the session status if it's available.
• If you have server-admin enabled, and have privileges, check the /admin page.
• Remote login to the session compute node and follow Linux instructions below.

For Desktop, Server, and Workbench on node where the session is running:

• Windows: Use the windows task manager to look for an rsession process
• Mac osx: Use the activity monitor to look for an rsession process
• Linux: Run this command using the session user's username: ps -flu username | grep rsession

If it is running, check the memory and CPU usage. Is the process using too much memory? Is it busy or idle?

When the rsession process is running, using CPU and not too much memory, it might be worth tracing the process to see what files it's accessing. You can use these tools: Linux: strace, Mac: dtrace, Windows: ProcessMonitor. In particular, look for open system calls to see the path names that rsession is accessing. Is it accessing unexpected files, or experiencing slow access times from using a network drive? Look for sendmsg and recvmsg calls to see snippets of network communication.

If it's running but appears to be stuck, force quit the process to see if that fixes the problem. You may also need to reset the session state in this case, or restart the computer.

Session output and logs

• Find the session output and look for errors.
• Find the log files and look for the rsession-username.log.
• Also look in rserver.log if using the Server or Workbench versions. By default it is in /var/log/rstudio/rstudio-server.

See if there are any errors such as:

• Segmentation violation/segfault, indicating that the R session crashed. Look for preceding errors that might explain the crash. Sometimes these errors are caused by conflicts in the versions of the shared libraries loaded. Use of strace or a similar tool point to what the program was doing before the crash. Was there an error returned from the operating system right before the crash? A particular file or socket connection being used that might add some insight.
• File permission errors. Check the associated files and see if the permissions are correct. The rsession runs with the account of the session user and so files and directories need to be accessible and writable by that user.
• Shared library loading errors.
• Network connection errors in the log can help trace which component is failing, and sometimes tell more about the failure. Look for your error in configuration problems.

Debug Logging.

When there are no clear errors or warnings, it may help to enable debug logging to get a more detailed log about what's happening in both the rserver and rsession.

Additionally, for problems that occur after the IDE has loaded the initial session, it's helpful to see the request flow from client to server that led up to the problem. There are three ways to get this information:

• In the IDE, go to the menu: Help -> Diagnostics -> Request Log. It will show the list of open connections, as well as provide access to the request and response messages exchanged between the IDE and the session. See this support article for more information.
• You can collect roughly this same info when using a browser by recording a HAR file.
• The rsession can also record a log of the request traffic it receives. Use the IDE's Tools -> Command Palette to enable Session protocol debug logging for the R session. This requires that debug logging for the session is enabled and output goes to the rsession-username.log log file.

Session failed to start

If rsession is failing to launch, look through these possible ways to fix the problem.

Antivirus software

Check antivirus logs, or explicitly configure it to allow permissions for sockets from RStudio Desktop to the R session and the browser to RStudio Server.

Firewall

If your system has a firewall enabled, make sure to open all of the ports necessary for your configuration, especially if load balancing.

Reset the user's session state

Sometimes the saved state for an R session can prevent it from starting up properly, or puts it into a state where a subsequent request might fail. It's at least a good data point to backup the session state, start from scratch, and see if the problem continues to occur. See the support article: Resetting a user's state.

Enable software rendering (Mac Desktop only)

A library used by RStudio Desktop can conflict with the default hardware rendering algorithm and cause Desktop to crash on startup. The workaround is to configure RStudio to do graphics rendering in software by running this command from the terminal, followed by a restart:

defaults write com.rstudio.desktop $'desktop\xb7renderingEngine' 'software'

Shared library errors

Identify the library that failed to load from the session output. Check the library path environment variable to be sure it's loading the correct version of R's main library as well as any packages used. Inspect installed libraries for compatibility with each other and the underlying OS. If you see errors about GLIBC version not found on linux, it typically means you are running an incompatible executable for the version of the OS. Check that you installed supported software for your operating system version.

One cause of errors is when the session user's environment or bash startup scripts customize the path to use for loading shared libraries in a way that conflicts with R, an R package, or the RStudio add ons. Each system has a different variable for the shared library path: - Linux: LD_LIBRARY_PATH - Mac: DYLD_LIBRARY_PATH - Windows: PATH

When the user's profile needs to customize the shared library path, a good practice is to add new directories to the end of the path rather than replacing it, or adding one to the front. For example on linux use this to add /opt/newProduct/lib:

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/newProduct/lib

This ensures RStudio's shared library path will not pick up unwanted versions of any libraries in the newly added directory.

It's also possible for two shared libraries to have conflicting dependencies - i.e. one requires a newer version of some 3rd library and the other requires an older version. Do you need to upgrade to a newer version of one of the packages, or downgrade another one? It can also occur if you are using a version of a package built for a newer version of a low-level system library like libc. On linux this generates an error like: version GLIBC_2.23 not found (required by rsession)

• Check the versions of the packages involved, and look for the versions those packages depend upon
• Check the package documentation of each for the system requirements. It can be helpful to look at the release dates of the packages to try and line up your dependencies to be roughly around the same time.
• On linux, you can use ldd and objdump to inspect this information on the executables and shared libraries when documentation is not available.

Configuration problems

By examining the logs, do you see errors that might be caused by any of these problems:

• Problematic R code in .RProfile, other startup scripts, or errors triggered when restoring the global environment. Try starting the session in safe mode
• An old rsession process is running but not responding, and preventing a new one from starting. Force quit the old rsession process and try restarting the session.
• Permission problems on the unix domain socket used for Desktop to session, or rserver to local rsession. The unix domain sockets for RStudio are stored under this path: /var/run/rstudio-server/rstudio-session/username-ds where username is the session user's username. Inside of that folder is one unix domain socket per session named by the session id.
• Out of disk space
• Session state corrupted due to misconfigured or unsupported file locking. RStudio uses file-locking to coordinate access to session metadata. It's important to eliminate race conditions that can occur in a load balanced environment, as well as to prevent multiple RStudio processes to read and write session metadata safely. If you see intermittent errors, use the locktester to validate that session locking is configured correctly for your file system.

Workbench with Slurm or Kubernetes

• Do the RStudio and R versions of session nodes running in Slurm/Kubernetes match the version installed on the RStudio Workbench node?
• Are the user's home and project directories available on both the server and session nodes using the same mount path?
• Can you use slurm/kubernetes to create a remote shell onto a compute node as the session user? Is there a difference between the su, ssh and rstudio-session PAM profiles? Use the pamtester to validate your configuration.

Slow session starting

When the R session is slow to start, eventually the IDE will give up on it and display an error. There are a number of potential causes for slow session starts:

• Out of memory restoring .RData or resuming a suspended session that's too large. Suspended session data is stored in ~/.local/share/rstudio/sessions/active/session-123/suspended-session-data (or check for the same path under your system's XDG_DATA_HOME environment variable).
• Slow loading of a large package index over a network file system. In this case, the R session eventually starts but it may take a long time. Try unchecking Enable packages pane in the Tools -> Global Options -> Packages menu to see if the session starts faster. Unchecking that option avoids a scanning process of packages in the R .libPaths that can result in a large number of file lookup requests to build the view of packages available.
• R code running in the profile, session start or resume process is slow, or using more memory than the system has or is exceeding a configured limit.

Try adjusting the rserver.conf setting rserver-proxy-max-wait-secs that controls how long to wait before responding with an error. The default is 10.

R session crashed after startup

Look in the session output for details about the crash. If it ran out of memory, increase the memory or try running with a smaller dataset. If it crashes running a specific program, try to narrow in on the package or request that's running using debug logs, the interactive debugger, or by tracing the process itself.

If it crashes at random times, check the available memory, use your system's debugger to get a stack trace of the crash to try and isolate the code involved. It may also be helpful to trace the process's system calls to see if there's a pattern of what it's doing when it crashes.

Some rsession crashes are due to R packages written in C or C++ that have bugs, and others are due to some interaction between R and the IDE. See if you can reproduce the crash in R standalone to isolate the problem.

Networking errors

Here's a list of potential causes for networking errors:

• A proxy server in the chain from the browser to RStudio is timing out long lived http connections, crashing, or otherwise interfering with the connection
• Antivirus software or firewall interfering with connections.
• Two independent RStudio Workbench clusters are sharing the same session user home directories. This can cause conflicts and unexpected behavior as they share a single directory to maintain active session metadata.
• Errors in rserver.log indicating stale message - caused by a Date HTTP header that is more than 60 seconds behind the RStudio Server's system time. This date is normally set by rserver but if your proxy layer is setting the date header, make sure the clocks on those systems are synchronized with the RStudio server.
• Errors caused by incorrect load balancing configuration
• Other errors in rserver.log that indicate network problems talking to the rsession
• Disrupted network connectivity

Finding session output

When the R session starts, the initial command output and error streams can be helpful in diagnosing problems, in particular if the session is crashing on startup.

Once R itself is started, most important messages are redirected to the R console and will not show up in the session output. If however, the R session crashes, an exit message describing the error goes to the original session output.

The output is found in different locations depending on how the session is started:

Desktop (Local)

When the Desktop is run locally, without connecting to a server the rsession process is started right from the RStudio desktop process and the output and error streams go right to the Desktop's standard output and error. To see these output streams, you will need to run RStudio from the command line, not by clicking on the icon.

Server/Workbench local sessions

For local R sessions run by RStudio Server or Workbench when not using launcher sessions, the initial output from running rsession goes to the standard output of the rserver process. If you launch rserver as a service, this output may not be captured in a log file or displayed anywhere. To see the output from a failed session start, you need to stop the service and run the rserver command from a terminal, or change the service configuration to capture the standard output of the command.

Workbench with launcher sessions

Normally the session output is available on the homepage by clicking on the Info button, then Details method in the Job associated with the session. If the session is not visible, or there's no job associated with the session the RStudio Workbench administrator can access the session output from the file system, where the location depends on the plugin used, the specific job-id and the username of the session user:

Local plugin

Look in the launcher's scratch-path, by default:

  /var/lib/rstudio-launcher/Local/output/username/jobid.{stdout,stderr}

Slurm plugin

Look in the session user's home directory in:

 ~/slurm-data/jobid.{stdout,stderr}

Kubernetes plugin

Look in the launcher's scratch-path, by default:

 /var/lib/rstudio-launcher/Kubernetes/output/username/jobid.{stdout,stderr}

Load balancing connection errors

In RStudio Workbench with load balancing enabled, check the load balancing configuration. Enable the debug logging and look in rserver.conf. If using the Job Launcher, make sure the scratch-path is set to a network file system shared by the nodes in the cluster and that the firewall permits each node to access the local port range of the other nodes.