why sys.stdout returned null

[yagweb]

Environment

• Pythonnet version: 2.2.2 (.NET 4, x64)
• Python version: 3.5
• Operating System: Windows 10

Details

Python is embedded in my .NET application using pythonnet. I found the output of the print statement in a python script is not showing on the screen until a "sys.stdout.flush()" called. I don't known how to redirect sys.stdout in .NET.
In order to call the flush() method directly in .NET, I tried

dynamic sys = PythonEngine.ImportModule("sys");
var stdout= sys.GetAttr("stdout"); // returns None

and both sys.stdout and the variable stdout returned are None here.

In python, I tried

import sys
print(getattr(sys, 'stdout'))

and it outputs a _io.TextIOWrapper object.

Why sys.stdout returned null here? How to make the output of python print statement show automatically?

Thanks.

[den-run-ai]

@yagweb can you try suggestions from here:

https://mail.python.org/pipermail/pythondotnet/2016-May/001777.html

http://stackoverflow.com/questions/37289082/redirect-stdout-stderr-from-python-embedded-in-net-application-to-text-box
https://mail.python.org/pipermail/pythondotnet/2016-May/001776.html

We should probably include some API to make this easier, since some people are already using this in ad-hoc ways.

[den-run-ai]

in your example make sure you acquire the GIL with using (Py.GIL()) {} and try this:

dynamic sys = PythonEngine.ImportModule("sys");
dynamic stdout = sys.stdout;

[yagweb]

@denfromufa Yes my code was put in the curly braces of '''using (Py.GIL()) {}'''. I have tried your code snippet, but the stdout is still null. I'm not sure whether it's a problem of CPython or pythonnet. Maybe somebody else can try it with the other versions of CPython to get more clues.

For redirecting stdout, 3 solutions are found according to these links, but the most elegant one (similar to the 3rd solution) provided by Cameron Hayne is not working because of the limitation of pythonnet. It needs “make any .NET object can be passed to CPython”. Here I give a briefly description, hope they can help with the API design.

Solution 1: Not automatically, so not suggested.

using (Py.GIL())
{
    PythonEngine.RunSimpleString("import sys\n"+
    "from io import StringIO\n"+
    "sys.stdout = mystdout = StringIO()\n");
    //test
    PythonEngine.RunSimpleString("print('Hello!')");
    Console.Write(stdout.getvalue());
}

Solution 2: Make Python call Console.Write automatically.

using (Py.GIL())
{
    PythonEngine.RunSimpleString(@"
import sys
from System import Console
class output(object):
    def write(self, msg):
        Console.Write(msg)
    def writelines(self, msgs):
        for msg in msgs:
            Console.Write(msg)
    def flush(self):
        pass
    def close(self):
        pass
sys.stdout = sys.stderr = output()
");
    //test
    PythonEngine.RunSimpleString("print('Hello!')");
}

Solution 3
This solution is similar to the one proposed by Cameron Hayne. Define a .NET class that implements the method expected of a Python file object, then create an instance and set it to sys.stdout.

using (Py.GIL())
{
    PythonEngine.RunSimpleString(@"
    import sys
    from Test import Output
    sys.stdout = sys.stderr = Output()
");
    //test
    PythonEngine.RunSimpleString("print('Hello!')");
}

public class Output
{
    public void write(String str)
    {
        Console.Write(str);
    }

    public void writelines(String[] str)
    {
        foreach (String line in str)
        {
            Console.Write(str);
        }
    }
    public void flush(){}
    public void close(){}
}

Solution 4: The best one but not working now.
A more elegent version of solution 3 proposed by Cameron Hayne looks like

Output output = new Output();
using (Py.GIL())
{
    dynamic sys = Py.Import("sys");
    sys.stdout = output;
    sys.stderr = output;
}

Howerver, Exception is throw when assigning sys.stdout because variable output is not of type PyObject. Make the Output class inherit from PyObject does not work. A possible solution is updating the TrySetMember method of PyObject as below. Make any .NET object can be passed to CPython is very attractive. Solution 3 may provide some help.

public override bool TrySetMember(SetMemberBinder binder, object value)
{
    if (this.HasAttr(binder.Name))
    {
         if (value is PyObject)
        {
            //some wrapping work. Can also be used for passing any .NET object as argument of PyFunction
            //Limited by my knowledge about pythonnet.
        }
        else
        {
            this.SetAttr(binder.Name, (PyObject)value);
            return true;
        }
     }
     else
    {
        return base.TrySetMember(binder, value);
    }
}

It’s worth to mention that, after set the sys.stdout in the three solutions, the stdout can be obtained normally

dynamic sys = Py.Import("sys");
var stdout = sys.stdout;   //now it‘s not null!

[filmor]

I like that last proposal a lot. Shouldn't be too difficult.

[den-run-ai]

@filmor @yagweb so you like this proposal to pass any .NET object to CPython without conversion? If yes, here is related issue:

#94

The 1st solution is what Python BDFL Guido recommends when embedding CPython:

https://docs.python.org/3/faq/extending.html#how-do-i-catch-the-output-from-pyerr-print-or-anything-that-prints-to-stdout-stderr

[yagweb]

@denfromufa @filmor I think this proposal is different from #94
This one talks about “pass any .NET object to CPython” and that one talks about “pass any CPython object to .NET”.
Both of them should take care about the reference management and efficiency.

I read the source code of converter.cs and found a way to do so. The PR has be committed. Several tests have conducted for this modification.

[den-run-ai]

@yagweb @filmor I like this 4th proposal as well. let's continue the relevant discussion in the pull request.

we can keep this issue open to find out why var stdout= sys.GetAttr("stdout"); // returns None.

[filmor]

We don't need to keep this open, this is documented behaviour:

https://docs.python.org/3/library/sys.html#sys.__stdin__

Note Under some conditions stdin, stdout and stderr [...] can be None. It is usually the case for Windows GUI apps that aren’t connected to a console and Python apps started with pythonw.

[yagweb]

@filmor @denfromufa I agree with this explanation. I have a proposal based on this mechanism.

When we embedded Python in a .NET Console application. Usually we want the Python print statement outputs to the screen directly.

So, what about the function PythonEngine.Initialize automatically redirects the sys.stdout and sys.stderr to the Console when it detecting the current application is a Console application just like the Python does, and the Solution 4 can be used here.

Or, add an argument "bool isredirect=false" to PythonEngine.Initialize, when the Initialize function is called with isredirect=true, it redirects the sys.stdout and sys.stderr to the Console using Solution 4.

[yagweb]

@filmor @denfromufa If the Solution 4 is chosen as the recommended way for the sys.stdout redirection. It's better to provide a .NET interface listing the methods needed. It can help us when we want the redirection in other senses.

[den-run-ai]

@yagweb i agree with this suggestion, and you @filmor?

Similar setup is provided for sys.argv in Initialize.

[filmor]

I agree with that, too. However, there doesn't seem to be a simple way to detect whether an application is running from a console in .NET. Also, the override should probably be done by wrapping Console.Out etc. instead of using the Console.Write functions.

[yagweb]

For any one who maybe interested in this topic, I add my PySysIO class here.

No automatically redirect, it can be used like this, or in this way.

// redirect to Console.Out, I usually put this at the entrance of my program.
PySysIO.ToConsoleOut();

//test
using (Py.GIL())
{
    PythonEngine.RunSimpleString("print('hello ConsoleOut!')");
}