# Designing a Paths Julep

**URL:** <https://discourse.julialang.org/t/designing-a-paths-julep/124335>\
**Category:** Internals & Design\
**Tags:** proposal, filesystem, path, rfc\
**Created:** [January 1, 2025, 12:37pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335 "2025-01-01T12:37:25Z")\
**Posts on this page:** 20\
**Page:** 6

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 13, 2025, 4:49pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/102 "2025-01-13T16:49:21Z")

</div>

> [@aplavin](#):
>
> To reduce confusion, you may want to clarify that in the proposal and clearly define the scope

I hoped (and still do) that:

> The case for `Path` as a fundamental type that belongs in Base Julia, and a proposal for implementing it well.

Along with the “Design goals” section makes the scope fairly clear, no?

This is still in the “being written” phase, so specific suggestions to make it clearer would be good 🙂

> [@aplavin](#):
>
> That’s what I tried to find in the proposal, but didn’t see any mention whatsoever.

Discussion of the abstract type is currently just the “High level path interface” subsection.

> [@aplavin](#):
>
> Now, it’s kinda-possible to write code that works with paths, and later apply it to remote files or files within archives. Is this intended to be possible within this proposal as well?

To make that work nicely, an abstract filesystem interface would probably be nice. This would be highly complementary but distinct from the abstract path interface. It’s not something I plan to investigate at this time, but perhaps you’d like to work on it?

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [January 13, 2025, 5:14pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/103 "2025-01-13T17:14:34Z")

</div>

> [@tecosaur](#):
>
> I hoped (and still do) that:
> 
> > The case for `Path` as a fundamental type that belongs in Base Julia, and a proposal for implementing it well.
> 
> Along with the “Design goals” section makes the scope fairly clear, no?

I still don’t get what exactly is the difference between the two of

> [@tecosaur](#):
>
> this is a _Path_ interface not a _Filesystem_ interface

in the Julia context.  
But if it’s clear to everyone else then of course feel free to keep it as is.

> [@tecosaur](#):
>
> Discussion of the abstract type is currently just the “High level path interface” subsection.

Representing anything beyond plain filesystem paths is a nontrivial concept, you may want to consider adding at least an explicit mention.

> [@tecosaur](#):
>
> To make that work nicely, an abstract filesystem interface would probably be nice. This would be highly complementary but distinct from the abstract path interface. It’s not something I plan to investigate at this time, but perhaps you’d like to work on it?

I’m _mostly_ fine with the current Julia interface, although would prefer non-string-based default path implementation if it existed.  
Consider me a _user_ of path interfaces (current or future), not a _developer_ of those 🙂

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 13, 2025, 5:17pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/104 "2025-01-13T17:17:02Z")

</div>

> [@aplavin](#):
>
> I still don’t get what exactly is the difference between the two of
> 
> > this is a _Path_ interface not a _Filesystem_ interface
> 
> in the Julia context.

There are a small number of operations one can do on a path not involving the filesystem (e.g. what is the parent directory, base name, extension, etc.), and then a larger pile of operations that involve the filesystem (e.g. `touch`, `mv`, `cp`, etc.). This proposal only involves an interface for the former.

---

<div class="post-metadata">

**Author:** ![Janis\_Erdmanis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/janis_erdmanis/32/10869_2.png) [@Janis\_Erdmanis](https://discourse.julialang.org/u/Janis_Erdmanis)\
**Post date:** [January 13, 2025, 7:38pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/105 "2025-01-13T19:38:09Z")

</div>

I see the problem then with the `parent` function that can return nothing. One can return an empty object instead. Then construction with joinpath no longer becomes ambigious.

I don’t see the benefit from deprecating `joinpath`, `dirname`, `basename` and `splitext` functions when working with path objects. By default I expect that we can sprinkle some `p` string macros and incrementally transition by returning `Path` from `pwd`, `@ __DIR__ `, `@ __FILE__ ` and other objects and expect the written code to work without changes.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 14, 2025, 3:50am UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/106 "2025-01-14T03:50:14Z")

</div>

> [@Janis\_Erdmanis](#):
>
> I don’t see the benefit from deprecating `joinpath`, `dirname`, `basename` and `splitext` functions when working with path objects. By default I expect that we can sprinkle some `p` string macros and incrementally transition by returning `Path` from `pwd`, `@ __DIR__ `, `@ __FILE__ ` and other objects and expect the written code to work without changes.

There’s no reason why `joinpath`, `basename` etc. wouldn’t work with `Path`s (in fact, `basename` is one of the proposed `AbstractPath` interface functions). I’m just that saying that “the parent of `"a.txt"` is `""`” seems a bit dodgy to me, when really there is no parent. It also means that you have paths where `dirname(p) == p` which IMO doesn’t make sense.

Regarding an incremental transition, `pwd`, `@ __DIR__ `, and `@ __FILE__ ` will need to continue to return `String` paths until Julia 2.0, as this is a breaking change. This doesn’t stop us from supporting `Path`s in key functions like `open` along with other Filesystem functions (`touch`, `mv`, `cp`, etc.), and we can push for the use of `Path`s over strings with these methods.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [January 14, 2025, 8:04am UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/107 "2025-01-14T08:04:38Z")

</div>

we could also just add `@ __PDIR__ ` and `@ __PFILE__ ` or so

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 14, 2025, 8:13am UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/108 "2025-01-14T08:13:12Z")

</div>

Yup, and similarly we could add `cwd() -> Path` as a `pwd()` analogue (which IMO is a misnomer, as it doesn’t “print” at all, so it’s just a vestigial shell-ism that reminds me of `car` and `cdr` in Lisp-land), and then once the path-based system is well established start `@depreciate`ing the String-path methods and encouraging people to use the relevant `Path` based one.

It’s a bunch of code churn, but:

- Old string-based code won’t stop working, and this shift can happen very gradually at whatever pace package authors are comfortable with
- There’s no easy way to avoid code churn, particularly when it comes to updating methods to accept `::Path` arguments
- We can make it easier by having good depreciations (NB: I think `@depreciate` could do with some work, IMO Emacs Lisp does it better ATM and is a notable example as it’s been highly backwards compatible for decades).
- Making the most of multiple dispatch, new `Path` methods for common functions will let a `String` → `Path` transition propagate through a call chain with minimal changes.

---

<div class="post-metadata">

**Author:** ![Janis\_Erdmanis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/janis_erdmanis/32/10869_2.png) [@Janis\_Erdmanis](https://discourse.julialang.org/u/Janis_Erdmanis)\
**Post date:** [January 14, 2025, 11:45am UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/109 "2025-01-14T11:45:42Z")

</div>

There is a school of thought that argues that returning an empty collection is better than returning a null value. If we look at a path as a list of strings, I see that the path fits in this category well.

One of the benefits is that error messages can be made more informative. For instance, consider the error message that is thrown to the user for`parent(parent(p“a.txt”))`. If we would return nothing, the best we could do is throw that the method does not exist. However, if we had an empty path passed to the parent, the error could have been made clear without introducing the `parent(::Nothing)` method.

Regarding the `dirname(p) == p`, we should throw an error at `dirname` for this reason. As I see it, one of the opportunities for introducing the `Path` type is to make existing code safer and make errors and warnings thrown early when the path is constructed instead of when system operations are executed.

> Regarding an incremental transition, pwd, @ **DIR** , and @ **FILE** will need to continue to return String paths until Julia 2.0, as this is a breaking change.

If the whole ecosystem continues to work as before when the `Path` type is returned instead, then I don’t see why we need to consider this a breaking change that requires Julia 2.0. Instead, I see this as a design constraint that can be addressed thanks to the flexibility of multiple dispatch. This also puts one on a realistic footing regarding whether the API is practical and does not require dirty conversions between path types or conversions to string and back.

I would want to see an analysis of each breaking change on how often a given pattern is used and what is a better alternative instead of doing a Platonic design here. For me, the new path system should fill the following design requirements:

- We shall have `AbstractPath` type that has a well-defined API
- The errors are thrown at the path construction time rather than at its use of executing methods
- The existing code shall work in as many instances as possible if `pwd`, `@ __DIR__ `, and `@ __FILE__ `, `joinpath` would return `::AbstractPath` type instead of `String`
- The string macro `p”...”` which I consider well considered in your proposal for some better usability
- Another design constraint is performance. It should be possible for users to write high-performance code using the `Path` type and, to some extent, nudge users to adopt better practices.

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [January 14, 2025, 8:04pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/110 "2025-01-14T20:04:52Z")

</div>

> [@davidanthoff](#):
>
> I think a base path type should be able to store and round-trip any path unnormalized that is considered valid by the platform.

[…]

> [@tecosaur](#):
>
> I’m still not convinced this _should_ be a supported use case. If you want an arbitrary sequence of characters to be used as a key, a `Path` type doesn’t sound like the most appropriate choice to me. Beyond dispatch, many of the benefits form having a path type come in the form of extra semanticics, and I don’t see nearly as much benefit in a path type without them.

In my mind paths are something where we are providing an abstraction in Julia for something that is defined by the OS platforms we run on, not a green room “what would an ideal path system look like”. In those cases, I think it is crucial that we stick to whatever (maybe horrible) platform design choices the folks that built the OS made, otherwise we run into the classical leaky abstraction problem: if we add a path type to Julia base that only can represent a subset of what is considered a valid path by the OS (or one that normalizes things in a way that is useful sometimes, but not always), well, then folks won’t be able to use that path type in all situations where a path is needed, and now we made the situation actually worse for them relative to not having anything: now one constantly needs to convert between path and string types, potentially store both etc.

I also think that base should (to the extent possible) not make calls a la “I think use-case A is valid for paths, but use-case B I don’t consider useful, so I’m not going to support it”. I think the right approach here is to try to support anything that one can do with whatever the OS considers a path. Someone will _always_ want to do that!

With this case of “we enforce normalization” I think it is not even close, the case of handling and dealing with non-normalized paths is not a niche situation, there are so many situations where one gets a path (from an API, from a file, whatever) and just wants to preserve it, I think a base path type really needs to support that.

I also actually don’t really understand what features wouldn’t be possible if non-normalized paths can be stored as such? Is there a concrete example of “if we support this, then we can’t provide feature X”?

Finally, I am _not_ saying that a path type that enforces normalization is not super, super useful in certain situations! It obviously is. But it strikes me as a more specialized scenario. Maybe something that could be handled in a NormalizedPaths.jl package, I don’t really think that needs to be in base.

Not having different types for Windows/Posix paths would actually make that much easier, something like [PathsAndURIs.jl/src/Paths/normalizedpath.jl at main · davidanthoff/PathsAndURIs.jl · GitHub](https://github.com/davidanthoff/PathsAndURIs.jl/blob/main/src/Paths/normalizedpath.jl) could then super easily work.

> [@davidanthoff](#):
>
> Another question where I’m not sure is whether it is actually good to have different concrete types for Windows and Posix paths, or whether it would be better to just have one type with some boolean flag or something like that.

> [@tecosaur](#):
>
> I’m not sure what’s best here, but it does seem nice to me that you can write filesystem methods for the current system with `::Path`. Otherwise all filesystem-interacting methods need to have a “is this path of the same kind of the system” error-path, and at a glance this is a stronger argument for making the system part of the type information than mixed system/non-system path arrays (unless I’m missing a major use case for mixed arrays?).

I don’t really think having an error path that checks whether a path is a valid path on the current system is so bad? Something like [`is_current_platform`](https://github.com/davidanthoff/PathsAndURIs.jl/blob/main/src/Paths/abstractpath.jl#L3) could be part of the path interface, and then calling that would just be part of the code at the beginning of a function that checks arguments for valid values. That seems a very common pattern?

The upsides to me seem really significant: if we use heterogenous types, then _any_ code that processes paths from multiple platforms will run into lots of dynamic dispatch problems, and those are really hard to get around. And that again doesn’t strike me as a niche scenario at all…

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [January 14, 2025, 8:36pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/111 "2025-01-14T20:36:13Z")

</div>

It sounds like such a path type wouldn’t be much of an abstraction anymore, but mostly a `String` called `Path` to have a dispatch type. While I get the argument that it’s annoying to have parts of OS path specs that you can’t represent faithfully with a path type, I personally value robust convenience functions that simplify the most common usage scenarios more. For example, I’m not sure if it’s bad if you can’t represent invalid paths even though they’re a subset of all paths that you can give to an OS to get a response from it. If that response is just always “this path is invalid” then what’s the point. You gave the example of dictionary keys above, but that again treats paths more like holistic strings and less like directions to something in a filesystem, where the directions you take and the target you end up at are more important than how the complete description of these directions looks (if there is more than one way to describe equivalent directions).

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [January 14, 2025, 9:13pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/112 "2025-01-14T21:13:51Z")

</div>

It would be nice if Paths could also be generalized for navigating data or for URIs.

For example, could we use it to navigate the following?

1. [XPath](https://developer.mozilla.org/en-US/docs/Web/XPath) queries for XML
2. [HDF5 hierarchies](https://support.hdfgroup.org/documentation/hdf5/latest/_h5_g__u_g.html) of groups and datasets
3. Some way to navigate JSON (JSONPaths or jq)

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 14, 2025, 9:22pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/113 "2025-01-14T21:22:05Z")

</div>

> [@davidanthoff](#):
>
> there are so many situations where one gets a path (from an API, from a file, whatever) and just wants to preserve it

I get the unfortunate sense we’re simply coming at this from rather different directions, but to me this use case really just sounds like a string named `Path`… but I won’t repeat Julius’ points, which I think are put rather well.

The one extra comment I’ll make is that regarding:

> [@davidanthoff](#):
>
> I don’t really think having an error path that checks whether a path is a valid path on the current system is so bad? Something like [`is_current_platform`](https://github.com/davidanthoff/PathsAndURIs.jl/blob/main/src/Paths/abstractpath.jl#L3) could be part of the path interface, and then calling that would just be part of the code at the beginning of a function that checks arguments for valid values. That seems a very common pattern?

This strikes me as an instance of “shotgun coding”, where a fragment of code needs to be injected into all sort of parts of a codebase and relies on the programmer “just remembering” to always get it right. I don’t like this on two accounts:

1. It relies on us fallible human programmers never forgetting to manually do the right checks in the right places.
2. It raises a run-time error and is thus much harder to statically detect, particularly in conditional code.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 14, 2025, 9:22pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/114 "2025-01-14T21:22:38Z")

</div>

> [@mkitti](#):
>
> It would be nice if Paths could also be generalized for navigating data or for URIs.
> 
> For example, could we use it to navigate the following?
> 
> 1. [XPath](https://developer.mozilla.org/en-US/docs/Web/XPath) queries for XML
> 2. [HDF5 hierarchies](https://support.hdfgroup.org/documentation/hdf5/latest/_h5_g__u_g.html) of groups and datasets
> 3. Some way to navigate JSON (JSONPaths or jq)

That’s my hope for the `AbstractPath` interface. Do let me know if you think it seems like enough 🙂

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [January 14, 2025, 10:00pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/115 "2025-01-14T22:00:49Z")

</div>

> [@jules](#):
>
> It sounds like such a path type wouldn’t be much of an abstraction anymore, but mostly a `String` called `Path` to have a dispatch type.

I don’t think that is correct. I’m not suggesting that we allow _any_ string value to be stored in a `Path`, all I suggest is that we allow storing of unnormalized paths. In such a design I believe _every_ structural point that was made above, for example that paths are collections of segments etc., still completely holds.

> [@jules](#):
>
> I personally value robust convenience functions that simplify the most common usage scenarios more.

I think this is a false dichotomy, I don’t believe allowing for storage of unnormalized paths would mean that we can’t have robust convenience functions. All the API proposals in the design spec doc and that were discussed here in the thread would still be there.

I think my question in my previous post is still useful: is there an example of something that could not be done in terms of API or something else in the proposal, if the path type would allow storage of unnormalized paths? I don’t see it right now, but maybe I’m missing something.

> [@jules](#):
>
> I’m not sure if it’s bad if you can’t represent invalid paths even though they’re a subset of all paths that you can give to an OS to get a response from it. If that response is just always “this path is invalid” then what’s the point.

Just to be clear, I am completely on board that a path type should _not_ be able to represent invalid paths.

> [@tecosaur](#):
>
> I get the unfortunate sense we’re simply coming at this from rather different directions,

And I think if something wants to go into base it needs to accommodate lots of use-cases 🙂 Otherwise I think a design is a great candidate for a package.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [January 15, 2025, 12:19am UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/116 "2025-01-15T00:19:23Z")

</div>

> [@jules](#):
>
> It sounds like such a path type wouldn’t be much of an abstraction anymore, but mostly a `String` called `Path` to have a dispatch type.

Even though this is an exaggeration as @davidanthoff points out, it doesn’t really feel fundamentally bad for me. It would already resolve the confusion whether one passes a path or a string with actual content. Anything else on top – any convenience or validation – are (useful) extras 🙂

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 15, 2025, 8:54am UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/117 "2025-01-15T08:54:14Z")

</div>

> [@aplavin](#):
>
> Anything else on top – any convenience or validation – are (useful) extras 🙂

Absolutely, past the brief `AbstractPath` section of the proposal, most of it is really just working out what’s worth putting on top. The WIP list (seen as subheaders) is:

- Avoid representational ambiguities
- Make invalid paths unconstructable
- Cross-platform path construction syntax
- Convenient prefixes
- Platform-specific path types
- Low overhead when invoking Libuv path methods
- SubStrings as the type of path components
- Iterable segments
- Safe path interpolation

> [@davidanthoff](#):
>
> I think it is crucial that we stick to whatever (maybe horrible) platform design choices the folks that built the OS made, otherwise we run into the classical leaky abstraction problem: if we add a path type to Julia base that only can represent a subset of what is considered a valid path by the OS (or one that normalizes things in a way that is useful sometimes, but not always), well, then folks won’t be able to use that path type in all situations where a path is needed

Just to be clear here, leaving normalisation aside, there’s no potential issue around not being able to represent a path on the system with the current design.

I do wonder whether it may be worth making an exception for the empty path though… which reminds me:

> [@aplavin](#):
>
> a neutral element

The empty path can’t be the neutral element, because the empty path can exist (with the right flags).

> [@davidanthoff](#):
>
> I also actually don’t really understand what features wouldn’t be possible if non-normalized paths can be stored as such? Is there a concrete example of “if we support this, then we can’t provide feature X”?

There are three key aspects that drive my enthusiasm for storing normalised forms:

1. Assuming a normalised form simplifies aspects of the implementation
2. Normalised forms lead to more sensible (IMO) behaviour in a number of respects
3. I liked Frame’s conceptual framework for paths, which is aided by normalised forms

To elaborate with some examples:

- Knowing that there’s only one segment separator character used makes finding segments in a path as easy as looking for the previous/next instance of that character.
- If we take to heart the “path as a series of directions” framing, it naturally follows that `C:\a/b` and `C:\a\b` should parse to the same path, the same way that the numbers `0.1` and `0.10` do — it’s purely a notational difference.
- Even if using paths as say dictionary keys, would we really want `/a/b` and `/a/./b` to map to different values? If the answer is “yes”, it really does seem to me as though you’re wanting to interpret the externally-supplied “path” as an arbitrary sequence of bytes (`String`), rather than a direction to a location on the filesystem (`Path`).

Maybe I am just missing the obvious, in which case an example of a use case where preserving notational differences in `Path`s is important and you also want to use a `Path` not a `String` would help me get on the same page as you.

---

<div class="post-metadata">

**Author:** ![jakobnissen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jakobnissen/32/13477_2.png) [@jakobnissen](https://discourse.julialang.org/u/jakobnissen)\
**Post date:** [January 15, 2025, 10:11am UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/118 "2025-01-15T10:11:30Z")

</div>

I also agree that we need to provide a convenient _abstraction_ over paths, not simply emulate its implementation in the OS.

From a broader perspective, one of the worst aspects of the Unix philosophy is that data should not be structured, but instead be simply text streams - the ‘universal interface’. The implementation and abstraction of paths that Unix provides is an embodiment of this terrible philosophy. In Unix, paths are trees that pretend they are byte arrays. It’s also the same kind of thinking that has some C programmers forego string types in favor of byte arrays.

I think modern programmers have mostly converged on the idea that types are a good thing, not because they enable more functionality, but because they guide the programmer towards writing robust code. We should lean heavily into this ideal and try to make a path type that, through its behaviour, signals how best to use it.  
That includes idea like automatic normalization.

However, there have been raised three counterpoints that limit the extend to which we can use the type system:

1. We need to support all existing path behaviours. That includes annoyances like invalid unicode paths, directories named `~`, and symlinks
2. Julia is a high-performance language. That limits the amount of automatic operations we can do on strings (if these operations are slow), and also limits how we can store strings - e.g. we probably shouldn’t store it as an array of individual segments.
3. The point raised by @kevbonham is an important issue. Lots of the things we want to do with paths are actually string operations. Do we add lots of string operations to paths, or do we ensure that one can easily and efficiently construct one type from the other? We probably need to try both things out and see how convenient they are, in practice.

For the latter point, I’m leaning towards _not_ implementation all the string operations, because

- It’s still relatively easy to do `Path(replace(String(path), "foo"=>"bar"))`
- Since Julia currently uses strings as paths, we need to continue supporting string anyway, so if users don’t want to opt-in to more type safety, they can keep using strings.
- For users to DO want to opt into type safety, those users probably don’t mind a little back and forth conversion in order to gain better static guarantees (i.e. no sneaky string operations they didn’t expect)
- It’s hard to consistently implement all string APIs for paths, so if we try, there will be a long period where it _usually_ works like a string but then just randomly fails because someone forgot to implement that one stirng operation for paths.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [January 15, 2025, 10:56am UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/119 "2025-01-15T10:56:27Z")

</div>

> [@jakobnissen](#):
>
> I think modern programmers have mostly converged on the idea that types are a good thing, not because they enable more functionality, but because they guide the programmer towards writing robust code. We should lean heavily into this ideal and try to make a path type that, through its behaviour, signals how best to use it.  
> That includes idea like automatic normalization.

Perhaps this is something that doesn’t adequately come across in the current document, but this is very much the way I’ve approached this effort. 💯

> [@jakobnissen](#):
>
> For the latter point, I’m leaning towards _not_ implementation all the string operations, because

I’m thinking that should we implement the functions like `withname`, `withsuffix`, etc. it should be similarly convenient over the status quo, but with added safety. E.g.

```julia
withname(mypath, replace(basename(mypath), "foo" => "bar"))

```

Since there’s a `Path(::Vector{String})` constructor, and `AbstractPath`s are iterable, for whole-path replacements, you could also do:

```julia
Path([replace(seg, "foo" => "bar") for seg in mypath])

```

and it should just work, with the bonus that `"bar"` can be an externally provided value, but without side-stepping the default constructor values that have special meaning would raise an error. This helps avoid the security hole created with code like:

```julia
function makehidden(path::Path, oldname::String, newname::String) # less-than-useful MWE demo function
    Path([replace(seg, oldname => ".$newname") for seg in path])
end

```

where somewhere upstream using `makehidden` ends up providing some sort of user/application/extenrally provided value as the `newname` argument, and the value `"."` is accidentally/maliciously passed. Equivalent code in Julia-of-today would cause a path segment to be replaced with `..` and hence change the path beyond what was likely intended/anticipated/safe, but with the current `Path`s prototype this instead raises a runtime error, and you have to do things a little more awkwardly if you want to allow special runtime-provided values to be injected into a path.

---

<div class="post-metadata">

**Author:** ![Janis\_Erdmanis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/janis_erdmanis/32/10869_2.png) [@Janis\_Erdmanis](https://discourse.julialang.org/u/Janis_Erdmanis)\
**Post date:** [January 15, 2025, 11:23am UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/120 "2025-01-15T11:23:53Z")

</div>

There would not seem to be much loss if filenames were to be represented as strings as they are not far from inputs to the `joinpath` function in practice. So we could use`basename(::AbstractPath)::String` and keep using `splitext(::String)::Vector{String}` function as before, except that one would need only add explicit `basename` in the code. One could add a compatibility method, `splitext(::AbstractPath)::Vector{String}`, which could give a warning but would still allow existing code to keep working.

> [@jakobnissen](#):
>
> It’s hard to consistently implement all string APIs for paths, so if we try, there will be a long period where it _usually_ works like a string but then just randomly fails because someone forgot to implement that one stirng operation for paths

It is hard, true, but that does not imply that we should not try. The new Path API can be shadowed on top of the Base exported methods, like `using PathLib; @pathify` where the `@pathify` macro can define shadowed methods that will be used instead. This enables running tests on registered packages with the modified API with a single regex replacement in the codebase `module` definition.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [January 15, 2025, 12:08pm UTC](https://discourse.julialang.org/t/designing-a-paths-julep/124335/121 "2025-01-15T12:08:41Z")

</div>

> [@jakobnissen](#):
>
> From a broader perspective, one of the worst aspects of the Unix philosophy is that data should not be structured, but instead be simply text streams - the ‘universal interface’.

That’s not the Unix philosophy. The Unix philosophy is:

- everything is a file
- the utility of an operating system/programming environment should come from being able to compose solutions well, instead of from a single monolithic solution

Text being prominent is reasonable though, it’s human readable. The drawback is having to parse repeatedly, but often that’s not an issue. In any case Unix is perfectly compatible with binary data.

> [@jakobnissen](#):
>
> The implementation and abstraction of paths that Unix provides is an embodiment of this terrible philosophy. In Unix, paths are trees that pretend they are byte arrays.

What do you mean? Can you propose an alternative?

[Previous page](https://discourse.julialang.org/t/designing-a-paths-julep/124335.md?page=5)

[Next page](https://discourse.julialang.org/t/designing-a-paths-julep/124335.md?page=7)