# How to break down string into chunk of n Kb while maintaining meaning
**URL:** https://discourse.julialang.org/t/how-to-break-down-string-into-chunk-of-n-kb-while-maintaining-meaning/107629
**Category:** General Usage
**Tags:** strings, aws
**Created:** [December 14, 2023, 8:16pm UTC](https://discourse.julialang.org/t/how-to-break-down-string-into-chunk-of-n-kb-while-maintaining-meaning/107629 "2023-12-14T20:16:19Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@Simone\_Gabbriellini](https://discourse.julialang.org/u/Simone_Gabbriellini)
#### Post date: [December 14, 2023, 8:16pm UTC](https://discourse.julialang.org/t/how-to-break-down-string-into-chunk-of-n-kb-while-maintaining-meaning/107629/1 "2023-12-14T20:16:19Z")
I am sending string to AWS Translate but I need to respect the 10000Kb limit for the string.
in case I have longer strings, I am using this approach
```julia
my_long_string = "some very long text...."
# AWS limit
aws_limit = 10000
# how many sentences we need
number_of_sentences = ceil(Base.summarysize(my_long_string) / aws_limit)
# break the string into an array of strings
list_of_sentences = segment_text(my_long_string, number_of_sentences)
```
now in my `segment_text` function I need to break down the string in `number_of_sentences`, and here I struggle a bit to understand the best way to cut the string using punctuation… the idea is to take a chunk as big as aws\_limit and break it to the nearest full stop (if any) before the end of the chunk, and progressively “consume” the whole `my_long_string`…
Is this by any chance a know algorithm for which already exists a solution?
---
### Author:  [@stevengj](https://discourse.julialang.org/u/stevengj)
#### Post date: [December 14, 2023, 11:45pm UTC](https://discourse.julialang.org/t/how-to-break-down-string-into-chunk-of-n-kb-while-maintaining-meaning/107629/2 "2023-12-14T23:45:22Z")
> [@Simone\_Gabbriellini](#):
>
> the idea is to take a chunk as big as aws\_limit and break it to the nearest full stop (if any) before the end of the chunk, and progressively “consume” the whole `my_long_string`…
For example, the following works in-place without copying any substrings (only uses views):
```julia
function process_chunks(process::Function, s::String, chunk_size::Integer, delimiter::Char='.')
# chunk size must be at least big enough for a single Unicode char
chunk_size ≥ 4 || throw(ArgumentError("chunk_size = $chunk_size is not ≥ 4"))
i = 1
e = lastindex(s)
@views while i <= e
# find maximum character index such that sizeof(s[i:j]) <= chunk_size
j = thisind(s, min(e, i + chunk_size-1))
nextind(s, j) > i + chunk_size && (j = prevind(s, j))
@assert j >= i
# shrink to last delimiter
j = i-1 + something(findlast(==(delimiter), s[i:j]), j-(i-1))
# process the chunk and continue
process(s[i:j])
i = nextind(s, j)
end
end
```
which gives:
```julia
julia> s = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.";
julia> process_chunks(println, s, 60)
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed
do eiusmod tempor incididunt ut labore et dolore magna aliq
ua.
Ut enim ad minim veniam, quis nostrud exercitation ullamco
laboris nisi ut aliquip ex ea commodo consequat.
Duis aute irure dolor in reprehenderit in voluptate velit e
sse cillum dolore eu fugiat nulla pariatur.
Excepteur sint occaecat cupidatat non proident, sunt in cul
pa qui officia deserunt mollit anim id est laborum.
```