Add Linux x64 Support #103

animusnull · 2019-12-18T01:34:30Z

This brings in support for linux x64. This is done via an upstream library zf log

This library will also work for android, iOs, mac os, etc. That being said I've only configured this for linux x64. I'm not familiar with Mac or mobile development.

Build Process Changes

As the library will not be present on most systems. It needs to be compiled and included. I have included a build script, and copied the source and header from the upstream repo with attribution to the initial authors.

Clarification Points

Configuration Items (How to configure this.)
- Log Level
- Format String (Left Blank for now)

oshai

Can you please also add documentation for the linux part of kotlin-logging

how to use it
how it works (briefly)
how to build it (the c parts)
You can simple add a readme file and later we will incorporate it with the main docs.

I reviewed it (except for the c files)
I will be glad if someone with native experience will review it as well. I will try to ask for help in kotlin slack.

oshai · 2019-12-18T07:37:43Z

src/linuxX64Main/kotlin/mu/KMarkerFactory.kt

+ */
+actual object KMarkerFactory {
+    actual fun getMarker(name: String): Marker {
+        TODO("not implemented") //To change body of created functions use File | Settings | File Templates.


please remove "To change body of ..." and change todo to UnsupportedOperationException (if you're not going to support it at the moment)

What is the marker for. I saw the interfaces for it. But wasn't entirely sure it's use case?

We used slf4j markers concept so I think quite a good explanation is here: https://stackoverflow.com/questions/16813032/what-are-markers-in-java-logging-frameworks-and-what-is-a-reason-to-use-them
If you want to take ideas from other implementations in kotlin-logging take a look at the js implementation as it is similar by the fact that it's not the original api the library was built with.

oshai · 2019-12-18T07:38:50Z

src/linuxX64Main/kotlin/mu/internal/KLogger.kt

+) : KLogger {
+
+    override fun entry(vararg argArray: Any?) {
+        TODO("not implemented") //To change body of created functions use File | Settings | File Templates.


same for TODO's in this file

I'll address those. First I want to clarify whether we should switch to platform snprintf or use the lib.

jnorthrup · 2019-12-18T08:14:03Z

adding an upstream c++ lib to do "a thin wrapper around snprintf() function." seems like it could just be a kotlin lib that is "a thin wrapper around snprintf() function." with greater streamlining in terms of system complexity

animusnull · 2019-12-18T08:49:51Z

adding an upstream c++ lib to do "a thin wrapper around snprintf() function." seems like it could just be a kotlin lib that is "a thin wrapper around snprintf() function." with greater streamlining in terms of system complexity

The big reason I chose this was it's linking for mobile. I'm relatively new to native, and have little iOs experience. so seeing a library with built in iOs support was what interested me. We can also do a simple snprintf or another print function. That's why I broke the appender out.

how to use it
how it works (briefly)
how to build it (the c parts)
You can simple add a readme file and later we will incorporate it with the main docs.
how to use it.

On a native only project.

Add a linuxX64 target. Add in the dependencies as noted below.

kotlin {
    linuxX64()
    sourceSets {
        linuxX64 { // Replace with a target you need.
            dependencies {
                implementation "io.github.microutils:kotlin-logging-linuxx64:1.7.9"
            }
        }
        linuxX64Test {}
    }

Compiling the C Library

My local development command structure has been.

➜  ~ cd ~/dev/kotlin-logging
➜  kotlin-logging git:(dev/linux_x64_native) ✗ cd src/nativeInterop/zfLog 
➜  zfLog git:(dev/linux_x64_native) ✗ ./build-linux.sh 
➜  zfLog git:(dev/linux_x64_native) ✗ cd ../../..
➜  kotlin-logging git:(dev/linux_x64_native) ✗ ./gradlew build publishToMavenLocal

This results in the following m2 repo tree. The prior name common conflicted with the gradle variant. I'm still trying to get it to work.

/home/sobrien/.m2/repository/io/github/microutils/
├── kotlin-logging
│   ├── 1.7.9
│   │   ├── kotlin-logging-1.7.9-javadoc.jar
│   │   ├── kotlin-logging-1.7.9.module
│   │   └── kotlin-logging-1.7.9.pom
│   └── maven-metadata-local.xml
├── kotlin-logging-js
│   ├── 1.7.9
│   │   ├── kotlin-logging-js-1.7.9.jar
│   │   ├── kotlin-logging-js-1.7.9-javadoc.jar
│   │   ├── kotlin-logging-js-1.7.9.module
│   │   ├── kotlin-logging-js-1.7.9.pom
│   │   └── kotlin-logging-js-1.7.9-sources.jar
│   └── maven-metadata-local.xml
├── kotlin-logging-jvm
│   ├── 1.7.9
│   │   ├── kotlin-logging-jvm-1.7.9.jar
│   │   ├── kotlin-logging-jvm-1.7.9-javadoc.jar
│   │   ├── kotlin-logging-jvm-1.7.9.module
│   │   ├── kotlin-logging-jvm-1.7.9.pom
│   │   └── kotlin-logging-jvm-1.7.9-sources.jar
│   └── maven-metadata-local.xml
├── kotlin-logging-linuxx64
│   ├── 1.7.9
│   │   ├── kotlin-logging-linuxx64-1.7.9-cinterop-zfLog.klib
│   │   ├── kotlin-logging-linuxx64-1.7.9-javadoc.jar
│   │   ├── kotlin-logging-linuxx64-1.7.9.klib
│   │   ├── kotlin-logging-linuxx64-1.7.9.module
│   │   ├── kotlin-logging-linuxx64-1.7.9.pom
│   │   └── kotlin-logging-linuxx64-1.7.9-sources.jar
│   └── maven-metadata-local.xml
└── kotlin-logging-metadata
    ├── 1.7.9
    │   ├── kotlin-logging-metadata-1.7.9.jar
    │   ├── kotlin-logging-metadata-1.7.9-javadoc.jar
    │   ├── kotlin-logging-metadata-1.7.9.module
    │   ├── kotlin-logging-metadata-1.7.9.pom
    │   └── kotlin-logging-metadata-1.7.9-sources.jar
    └── maven-metadata-local.xml

The linuxX64 contains several klibs which will recompile on any project pulling in depending on it. This is one of the reasons a lot of project also do host detection to determine how to compile the project.

Multi Source Set Library

If you are consuming it in a project with targets outside of linuxX64 i.e. js, jvm, common, etc. You need to define an additional native tool set. I have not determined kts syntax so the following is groovy/gradle dsl.

kotlin {
    jvm()
    js {}
    linuxX64() {}
    sourceSets {
        commonMain {
            dependencies {
                implementation "io.github.microutils:kotlin-logging:$KotlinLoggingVersion"
                implementation kotlin('stdlib-common')
            }
        }
        commonTest { }
        jvmMain {
            dependencies {
                implementation kotlin('stdlib-jdk8')
                implementation "io.github.microutils:kotlin-logging:$KotlinLoggingVersion"
            }
        }
        jvmTest { }
        jsMain {
            dependencies {
                implementation "io.github.microutils:kotlin-logging-js:$KotlinLoggingVersion"
                implementation kotlin('stdlib-js')
            }
        }
        jsTest { }
        nativeMain {
            dependsOn commonMain
        }
        linuxX64Main { // Replace with a target you need.
            dependsOn nativeMain
            dependencies {
                implementation "io.github.microutils:kotlin-logging-linuxx64:$KotlinLoggingVersion"
                implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime-native:$SerializationVersion"

            }
        }
        linuxX64Test { }
    }
}

** Usage **

From a native project it's not much different than the existing implementation.

    override val logger: KLogger = KotlinLogging.logger {}
    logger.error { "Failed to connect got back null, reason unknown. Dying in a fire." }

Notes

If we stick with zf_log, I am totally ok with redoing this as a print function. There is a format string that needs to be set.
The log level also needs to be set.

How do we want to set those two variables. Environment variables, text file configuration, or something else.

Upstream link to a sample file

…o mutability issues.

animusnull · 2019-12-20T00:50:47Z

Deprecating zfLog. That should be fully removed. I introduced a new abstract class. Which in the toString calls makeFormat. This allows us to build a format string via a series of objects. I have base line elements of timestamp, month, day, year, etc. Time is in 24 hours and gmt/utc.

The configuration was switched from an object -> data class. This is due to the invalid mutability exception. Recommended reading on the topic

The message will be

[$LogLevel]${format string} $message

The log level will always be the first item printed. The default format string is.

val format: String = "[${Month}/${Day}/${Year}][${Hour}:${Minute}:${Second} ${TimeZone}]"

This approach allows us to easily add new elements as needed to native. Be it process name, thread, etc.

A sample implementation

fun main() {
    val logger = KotlinLogging.logger(KotlinLoggingConfiguration(logLevel = Debug)) {}
    logger.info { "INFO" }
    logger.debug { "DEBUG" }
    logger.trace { "TRACE" }
    logger.warn { "WARN" }
    logger.error { "ERROR" }
    val newLogger = KotlinLogging.logger(KotlinLoggingConfiguration(logLevel = Debug, format = "$TimeStamp ")) {}
    newLogger.info { "INFO" }
    newLogger.debug { "DEBUG" }
    newLogger.trace { "TRACE" }
    newLogger.warn { "WARN" }
    newLogger.error { "ERROR" }
}

Corresponding logs

[Info][12/20/2019][0:42:03 GMT]INFO
[Debug][12/20/2019][0:42:03 GMT]DEBUG
[Warn][12/20/2019][0:42:03 GMT]TRACE
[Warn][12/20/2019][0:42:03 GMT]WARN
[Info]1576802523673 INFO
[Debug]1576802523673 DEBUG
[Warn]1576802523673 TRACE
[Warn]1576802523673 WARN

Dominaezzz

Saw the comment about wanting a review.

Dominaezzz · 2019-12-25T16:43:08Z

build.gradle.kts

+    val linuxX64 by sourceSets.creating {
+        dependsOn(nativeMain)
+    }
+    targets.withType(KotlinNativeTarget::class.java) {


This can just be targets.withType<KotlinNativeTarget> {.

Dominaezzz · 2019-12-25T16:43:57Z

build.gradle.kts

@@ -67,6 +79,16 @@ kotlin {
            }
        }
    }
+    val linuxX64 by sourceSets.creating {


Creating this source set isn't necessary, since the target will create one already.

The problem I had was referencing via dependsOn. I was having trouble where this project wouldn't work in a multi platform build. Without creating a nativeMain, and setting dependsOn, which requires a source set. The presets from konan I tried wasn't working. I realize it's duplicated code but couldn't find a work around.

val linuxX64 by sourceSets.creating { dependsOn(nativeMain) } ... "linuxX64" -> compilations["main"].defaultSourceSet.dependsOn(linuxX64)

Hmm, does auto-complete/intelli-sense work in the IDE as it is?
If it doesn't then compilations["main"].defaultSourceSet.kotlin.srcDir("src/nativeMain/kotlin") should make it work. Otherwise it doesn't really matter I guess.

KTS results in a bunch of errors in IntelliJ for me. I've tried wiping my settings a number of times but haven't gotten it to work. Essentially no auto complete, or type hinting. Running Intellij Community Latest.

Dominaezzz · 2019-12-25T16:45:56Z

nativeHelper.gradle.kts

@@ -0,0 +1,11 @@
+val hostOs = System.getProperty("os.name")


You can replace all of this with the HostManager class, which comes with the kotlin gradle plugin. You can then do stuff like, HostManager.isMingw and HostManager.isLinux.

Changed over thanks

Dominaezzz · 2019-12-25T16:50:04Z

src/linuxX64Main/kotlin/mu/internal/ConsoleAppender.kt

+
+object ConsoleAppender : IAppender {
+    private val STDOUT = platform.posix.fdopen(1, "w")
+    private val STDERR = platform.posix.fdopen(2, "w")


Instead of explicitly opening a descriptor, can you use platform.posix.stderr and platform.posix.stdout?

Dominaezzz · 2019-12-25T16:53:44Z

src/linuxX64Main/kotlin/mu/internal/Formatter.kt

+ * Gets a date struct, [upstream docs](http://www.cplusplus.com/reference/ctime/tm/)
+ */
+fun getDateStruct() = memScoped {
+    val t = alloc<time_tVar>()


val t = cValue<time_tVar> { value = time(null) } is an option. Then you don't need memScoped.

I had seen a similar snippet at one point.

Using

val t = cValue<time_tVar> { value = time(null) } // Gave Type argument is not within its bounds: should be subtype of 'CStructVar' // For <time_tVar>

Ah! Nvm then.

oshai · 2019-12-29T07:28:06Z

@animusnull - whats is the status of the PR from your side?

animusnull · 2019-12-29T21:37:52Z

@animusnull - whats is the status of the PR from your side?

I had missed the other feed back by @Dominaezzz
I will work on those comments and push the changes up tonight or tomorrow. Then it would just be focusing on the build process and artifacts.

… built std[err|out] handlers.

animusnull · 2019-12-30T00:52:31Z

Just pushed changes. I still couldn't knock away the explicit linuxX64 source set, and couldn't deprecate memscoped.

Removed native target helper, moving most remaining items into main build.gradle.kts. Using hostmanager.

oshai · 2020-02-06T07:30:52Z

@animusnull - from your perspective this should be merged? or is there anything else you want to add?

animusnull · 2020-02-14T00:50:07Z

Linux x64 works. I have it building via a fork and publishing artifacts.

But there are a couple of catches.

Opening Intellij on anything other than Linux causes it to not build. Due to the on else HostManager throw error.
CI/CD is another question, and ensuring it builds properly. The base container image can sometimes cause problems with Konan the Kotlin native builder.

From a code stand point, I think it's good. It's more CI/CD integration. Maybe take the time to build for other platforms.

I can take on Windows, but I don't have a iOs, Mac Os device.

initial push.

a9f83ee

oshai mentioned this pull request Dec 18, 2019

Kotlin/Native Support #45

Closed

oshai reviewed Dec 18, 2019

View reviewed changes

Bug fixes for multi platform projects.

68a2a37

animusnull added 2 commits December 19, 2019 11:18

Deprecating zfLog, adding base console appender.

9000a48

Adding in formatter. Switching config from object -> data class due t…

d9793c7

…o mutability issues.

Dominaezzz reviewed Dec 25, 2019

View reviewed changes

Deprecating native helper, bringing in host manager. Switching to pre…

9cb6255

… built std[err|out] handlers.

animusnull closed this Jun 21, 2020

napperley mentioned this pull request Jun 21, 2020

Add Linux X64 Support #119

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Linux x64 Support #103

Add Linux x64 Support #103

animusnull commented Dec 18, 2019

oshai left a comment

oshai Dec 18, 2019

animusnull Dec 18, 2019

oshai Dec 18, 2019

oshai Dec 18, 2019

animusnull Dec 18, 2019

jnorthrup commented Dec 18, 2019

animusnull commented Dec 18, 2019

animusnull commented Dec 20, 2019

Dominaezzz left a comment

Dominaezzz Dec 25, 2019

animusnull Dec 30, 2019

Dominaezzz Dec 25, 2019

animusnull Dec 30, 2019

Dominaezzz Dec 30, 2019

animusnull Jan 2, 2020

Dominaezzz Dec 25, 2019

animusnull Dec 30, 2019

Dominaezzz Dec 25, 2019

animusnull Dec 30, 2019

Dominaezzz Dec 25, 2019

animusnull Dec 30, 2019

Dominaezzz Dec 30, 2019

oshai commented Dec 29, 2019

animusnull commented Dec 29, 2019

animusnull commented Dec 30, 2019

oshai commented Feb 6, 2020

animusnull commented Feb 14, 2020

Add Linux x64 Support #103

Add Linux x64 Support #103

Conversation

animusnull commented Dec 18, 2019

oshai left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnorthrup commented Dec 18, 2019

animusnull commented Dec 18, 2019

animusnull commented Dec 20, 2019

Dominaezzz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oshai commented Dec 29, 2019

animusnull commented Dec 29, 2019

animusnull commented Dec 30, 2019

oshai commented Feb 6, 2020

animusnull commented Feb 14, 2020