# Splitpath/joinpath for non-native paths?

**URL:** <https://discourse.julialang.org/t/splitpath-joinpath-for-non-native-paths/123336>\
**Category:** General Usage\
**Tags:** filesystem\
**Created:** [December 2, 2024, 2:23am UTC](https://discourse.julialang.org/t/splitpath-joinpath-for-non-native-paths/123336 "2024-12-02T02:23:44Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [December 2, 2024, 2:23am UTC](https://discourse.julialang.org/t/splitpath-joinpath-for-non-native-paths/123336/1 "2024-12-02T02:23:44Z")

</div>

> [@Convert filepath to URI](https://discourse.julialang.org/t/convert-filepath-to-uri/123334/4):
>
> It won’t convert backslash to forward slashes

This problem highlights a problem in Base.Filesystem. The natural way to do this conversion would be to use something along this line:

```julia
joinpath(
  splitpath("a\\b\\c"; arch=Base.Arch.Windows); 
  arch=Base.Arch.Linux
) == "a/b/c"

```

The underlying string transformations do not depend on the architecture they are running in. This situation is akin to algorithm selection in `sort`.

Currently, Base.Filesystem `joinpath` and other functions use `Sys.iswindows()` and `Sys.isunix()` to query the current system. This is nice default behaviour, but it is obvious people need access to the other transformation functions.

(this comment not directly related to OP, so moderators can feel free to split it to different thread)

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [December 2, 2024, 2:54am UTC](https://discourse.julialang.org/t/splitpath-joinpath-for-non-native-paths/123336/2 "2024-12-02T02:54:31Z")

</div>

> [@Convert filepath to URI](https://discourse.julialang.org/t/convert-filepath-to-uri/123334/4):
>
> > [@Convert filepath to URI](https://discourse.julialang.org/t/convert-filepath-to-uri/123334/4):
> >
> > It won’t convert backslash to forward slashes
> 
> This problem highlights a problem in Base.Filesystem.

Well, the “it” in my sentence referred to `URIs.escapeuri` — URIs.jl currently doesn’t have any functions for handling file paths, _only_ URI paths, so this particular issue had nothing to do with `Base`.

Regarding your point, though, that `splitpath` and `joinpath` are currently only for the system you are running on, I agree that in _principle_ it would be nice to be able to handle “foreign” file paths too. Though this doesn’t seem to come up in practice very often?

(However, it shouldn’t be an issue for @thestoicone in ~~this thread~~ the original thread, because they are using `walkdir` to generate file paths, and hence their paths will always be for the native OS.)

---

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [December 2, 2024, 3:36am UTC](https://discourse.julialang.org/t/splitpath-joinpath-for-non-native-paths/123336/3 "2024-12-02T03:36:52Z")

</div>

> [@stevengj](#):
>
> in _principle_

There is another principle at work here and it is function purity. Given a parametrized specification of the OS makes the functions pure. Pure functions are easier to reason about and to formally handle.

Although the quick default of using current system needs to be there. A good _principle_ would be for functions to be easily modified into pure functions from the caller’s location.

This essentially is why we like the ability to specify an RNG for functions. Impurity and stochasticity should be easily turned off.

Going further, even better, an automated tool which fixes such parameters (OS and RNG) and drives functions to be as pure as possible and warns of possible randomness or impurity could be extra good.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [December 2, 2024, 5:46am UTC](https://discourse.julialang.org/t/splitpath-joinpath-for-non-native-paths/123336/4 "2024-12-02T05:46:35Z")

</div>

For fun, I’ll mention the recent outcome of a #gripe in Slack: there’s interest in doing take 2 of a Julep adding a path type to Base. The current design includes `PosixPath` and `WindowsPath` concrete types, and so you would be able to reason about non-native paths in this way.

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [December 2, 2024, 6:04am UTC](https://discourse.julialang.org/t/splitpath-joinpath-for-non-native-paths/123336/5 "2024-12-02T06:04:34Z")

</div>

I played around with an alternative URI and path type design a while ago, completely forgot about it. But I just uploaded to GitHub, the types are at [URIs2.jl/src/types.jl at main · davidanthoff/URIs2.jl · GitHub](https://github.com/davidanthoff/URIs2.jl/blob/main/src/types.jl), and the README has some more thoughts on the design ([GitHub - davidanthoff/URIs2.jl](https://github.com/davidanthoff/URIs2.jl)).

One thing that is different in this design (right now only visible in the URI parts) is that I’m using types to distinguish different storage layouts, rather than semantic differences. I think for a path type design I would probably go the same direction these days, i.e. something like `AbstractPath`, and then `Path` stores things just as one string, and `PathParts` stores it as tuple that holds the parts, or something like that. I would store the semantic difference between windows and posix parts just as a `Bool` flag.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [December 2, 2024, 8:30am UTC](https://discourse.julialang.org/t/splitpath-joinpath-for-non-native-paths/123336/6 "2024-12-02T08:30:48Z")

</div>

Seems like more people than I thought have done experiments in this direction 👀. Do you think I could interest you in the latest (WIP) Path type proposal David? It currently has this type hierarchy:

```julia
AbstractPath{T}
└── PlainPath (<: AbstractPath{SubString{String}})
    └── SystemPath
        ├── PosixPath
        └── WindowsPath

```

And a 6-function interface (`root`, `parent`, `basename`, `iterate`, `length`, `*`).

There are also some windows particularities that I’m vaguely aware of, but not familiar with (UNC paths, shares, and other fun stuff), that might necessitate some tweaks to the design.

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [December 2, 2024, 5:33pm UTC](https://discourse.julialang.org/t/splitpath-joinpath-for-non-native-paths/123336/7 "2024-12-02T17:33:34Z")

</div>

I think the main question I have these days is whether using the types to distinguish between windows and posix paths is the right choice. At the moment I’m more tempted to use types to distinguish between different memory layouts for storing a path, and making the windows/posix distinction just a plain field in the type. So something like:

```julia
abstract type AbstractPath end

struct Path <: AbstractPath
  _internal::String
  _windows::Bool # Or maybe an enum or Symbol
end

struct PathParts <: AbstractPath
  _parts::Tuple{Vararg{String}}
  _windows::Bool
end

```

I see at least two benefits of this: First, for different use-cases different memory layouts are better, so I generally think that something that locks this down to one choice is not great. Second, I think with a type hierarchy that is based on windows/posix there is a fair chance that one ends up with heterogeneous arrays, dynamic dispatch etc., and so to me right now it seems easier to just encode whether something is a Windows path as a field.

Finally, I would probably keep it simple, and just say “this is for file paths”, and not try to design something that also covers all sorts of other paths. I think the latter leads to a fairly complicated design, with not much upside. If someone wants to add support for say S3 paths, or something like that, they can just create a new type, I don’t see a really good reason why all of that needs to be part of one type hierarchy.

> [@tecosaur](#):
>
> Do you think I could interest you in the latest (WIP) Path type proposal David?

Do you have a link? Happy to take a look.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [December 2, 2024, 5:39pm UTC](https://discourse.julialang.org/t/splitpath-joinpath-for-non-native-paths/123336/8 "2024-12-02T17:39:06Z")

</div>

> [@davidanthoff](#):
>
> At the moment I’m more tempted to use types to distinguish between different memory layouts for storing a path

I don’t see why the memory layout should be different?

> [@davidanthoff](#):
>
> Second, I think with a type hierarchy that is based on windows/posix there is a fair chance that one ends up with heterogeneous arrays, dynamic dispatch etc., and so to me right now it seems easier to just encode whether something is a Windows path as a field.

I have trouble seeing many situations where you’d want to feed a path for a different platform into a function, so I’d expect most functions to just be defined for the current platform type (except for basic abstract path manipulations that have definitions for both) — and so I don’t see much of a risk of heterogeneous arrays, dynamic dispatch, etc.

> [@davidanthoff](#):
>
> If someone wants to add support for say S3 paths, or something like that, they can just create a new type

Sure, but I do think it’s nice to provide an appropriate abstract type to subtype.

> [@davidanthoff](#):
>
> Do you have a link? Happy to take a look.

I do, this is currently very much WIP though (half-finished, started last weekend), so I’ll probably just DM it to you rather than share it publicly yet.

---

<div class="post-metadata">

**Author:** ![JamesNZ](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jamesnz/32/35361_2.png) [@JamesNZ](https://discourse.julialang.org/u/JamesNZ)\
**Post date:** [December 2, 2024, 11:21pm UTC](https://discourse.julialang.org/t/splitpath-joinpath-for-non-native-paths/123336/9 "2024-12-02T23:21:16Z")

</div>

> [@tecosaur](#):
>
> I have trouble seeing many situations where you’d want to feed a path for a different platform into a function, so I’d expect most functions to just be defined for the current platform type (except for basic abstract path manipulations that have definitions for both) — and so I don’t see much of a risk of heterogeneous arrays, dynamic dispatch, etc.

Just wanted to mention that one use-case is working with remote machines of a different platform. e.g. for the sake of ~~laziness~~ simplicity LibSSH.jl’s SFTP implementation currently assumes that the server is always running on \*nix:

- [SFTP · LibSSH](https://juliaweb.github.io/LibSSH.jl/stable/sftp/)
- [LibSSH.jl/src/utils.jl at ffe521caca5cdefcd0646b3529020fad167a8e5b · JuliaWeb/LibSSH.jl · GitHub](https://github.com/JuliaWeb/LibSSH.jl/blob/ffe521caca5cdefcd0646b3529020fad167a8e5b/src/utils.jl#L17)

(though I personally don’t have a strong opinion on whether it should be a type or field as long as `joinpath()` does the right thing)

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [December 2, 2024, 11:44pm UTC](https://discourse.julialang.org/t/splitpath-joinpath-for-non-native-paths/123336/10 "2024-12-02T23:44:40Z")

</div>

> [@tecosaur](#):
>
> I have trouble seeing many situations where you’d want to feed a path for a different platform into a function

Log file processing, or essentially _any_ file that might embed a filename that could have been generated on a different platform. This to me seems a _very_ common thing.