JDK-8302160 : JDK 20+30 breaks external JLine usage with Error executing 'tty': not a tty (java.io.IOException)
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.io
  • Affected Version: 20
  • Priority: P3
  • Status: Resolved
  • Resolution: Cannot Reproduce
  • Submitted: 2023-02-09
  • Updated: 2023-02-09
  • Resolved: 2023-02-09
Related Reports
Relates :  
JDK-8299687 - Modify JLine library not to spawn native processes
Relates :  
JDK-8295803 - Console should be usable in jshell and other environments
Description
I found a regression between JDK 20+26 and JDK 20+30.
Before, these test work fine, after they fail like:
```
Couldn't initialize readline (java.lang.UnsupportedOperationException)
	from org.truffleruby.stdlib.readline.ConsoleHolder.<init>(ConsoleHolder.java:142)
	from org.truffleruby.stdlib.readline.ConsoleHolder.create(ConsoleHolder.java:89)
	from org.truffleruby.RubyContext.getConsoleHolder(RubyContext.java:677)
	from org.truffleruby.stdlib.readline.ReadlineNodes$BasicWordBreakCharactersNode.basicWordBreakCharacters(ReadlineNodes.java:69)
	from org.truffleruby.stdlib.readline.ReadlineNodesFactory$BasicWordBreakCharactersNodeFactory$BasicWordBreakCharactersNodeGen.execute(ReadlineNodesFactory.java:105)
	from org.truffleruby.language.RubyCoreMethodRootNode.execute(RubyCoreMethodRootNode.java:58)
Caused by:
Not a tty (java.io.IOException)
	from org.graalvm.shadowed.org.jline.terminal.impl.ExecPty.current(ExecPty.java:44)
	from org.truffleruby.stdlib.readline.ConsoleHolder.<init>(ConsoleHolder.java:127)
	from org.truffleruby.stdlib.readline.ConsoleHolder.create(ConsoleHolder.java:89)
	from org.truffleruby.RubyContext.getConsoleHolder(RubyContext.java:677)
	from org.truffleruby.stdlib.readline.ReadlineNodes$BasicWordBreakCharactersNode.basicWordBreakCharacters(ReadlineNodes.java:69)
	from org.truffleruby.stdlib.readline.ReadlineNodesFactory$BasicWordBreakCharactersNodeFactory$BasicWordBreakCharactersNodeGen.execute(ReadlineNodesFactory.java:105)
	from org.truffleruby.language.RubyCoreMethodRootNode.execute(RubyCoreMethodRootNode.java:58)
Caused by:
Error executing 'tty': not a tty (java.io.IOException)
	from org.graalvm.shadowed.org.jline.utils.ExecHelper.exec(ExecHelper.java:42)
	from org.graalvm.shadowed.org.jline.terminal.impl.ExecPty.current(ExecPty.java:41)
	from org.truffleruby.stdlib.readline.ConsoleHolder.<init>(ConsoleHolder.java:127)
	from org.truffleruby.stdlib.readline.ConsoleHolder.create(ConsoleHolder.java:89)
	from org.truffleruby.RubyContext.getConsoleHolder(RubyContext.java:677)
	from org.truffleruby.stdlib.readline.ReadlineNodes$BasicWordBreakCharactersNode.basicWordBreakCharacters(ReadlineNodes.java:69)
	from org.truffleruby.stdlib.readline.ReadlineNodesFactory$BasicWordBreakCharactersNodeFactory$BasicWordBreakCharactersNodeGen.execute(ReadlineNodesFactory.java:105)
	from org.truffleruby.language.RubyCoreMethodRootNode.execute(RubyCoreMethodRootNode.java:58)
```

I would guess it's related to the recent addition of the internal JLine.

That error seems to mean the `tty` command had output `not a tty`.
Locally on linux-amd64, I get that output when stdin is /dev/null: `tty </dev/null`, but not if stdout and/or stderr are redirected (with original stdin).

I'm not sure how but it seems the new internal JLine messes up existing usages of JLine.
Maybe there can only be a single system terminal, or there is some side effect of changing stdin/stdout/stderr when calling `System.console() != null` (to find out if a TTY)?

Given the various issues with the internal JLine (e.g. JDK-8299689),
maybe it is worth reconsidering whether it's a good idea to add an internal copy of JLine in the JDK?

You can reproduce with the instructions to build TruffleRuby at https://github.com/oracle/truffleruby/blob/master/doc/contributor/workflow.md
And then:
```
bin/jt test spec/ruby/library/readline
```

This works fine on JDK 20+26, but fails on JDK 20+30.

It happens on at least linux-amd64, darwin-amd64 and darwin-aarch64.
Comments
This issue is not reproducible with the fix to JDK-8299689 as the JLine is now the opt-in, the default Console is using the built-in console that is compatible with the old JDK releases. Note that `jdk.console` is not a public system property as of now. Having said that, it is correct that enabling JLine without tty availability needs consideration. It will be resolved in JDK-8299687
09-02-2023
Console usages is stifled because it is not usable in IDEs, jshell, and other virtual terminal environments. You are right that there isn't an isatty equivalent, code needs to the JNI or the FFM API to call that. The only JBS issue request isatty is from 1997, I couldn't find any other requests for it but it could be looked at when the issue of Console working with virtual terminals is looked at again.
09-02-2023
Reading the original ticket, I think the bug is pretty simple: `System.console()` is never null with the JLine System.console, and as a result any usage which relies on `System.console() != null` as a isatty(3) check is broken. https://bugs.openjdk.org/browse/JDK-8295803?focusedCommentId=14558878&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14558878 The test failure above is then because the test checks a case where it's not a TTY (stdin is a file), but `System.console() != null` is true and so it tries to create a JLine system Terminal for it, but that just fails given there is no TTY.
09-02-2023
The intention is to have Console work with virtual terminals, in IDEs, jshell, etc. Also have line editing / recall helps the user experience. The reason it's not the default in JDK 20 is because jline launches programs such as tty to get console characteristics - this part needs to replaced to avoid that.
09-02-2023
Indeed, -Djdk.console=java.base workarounds it. I have also found that using a newer JDK which has JDK-8299689 workarounds the problem. And that setting `-Djdk.console=jdk.internal.le` makes the problem happen again. So at least latest JDK 20 doesn't repro this by default, but it is still an issue for anyone using `-Djdk.console=java.base` or if the default would change to that in the future. Maybe the `System.console() != null` way to detect if a TTY is attached is problematic/unfortunate for this case. Is there any replacements? It'd be nice if the JDK would expose a way to know if stdin/stdout/stderr are isatty(3), that would notably avoid creating a console just to find that out (and as we see here that has non-trivial side effects).
09-02-2023
The package name is org.graalvm.shadowed.org.jline.terminal.impl so the issue reported here may be due to using two versions of jline at the same time. It's possible that running with -Djdk.console=java.base will workaround this.
09-02-2023