How to parse multiline string in Julia?
It's a bit dopey, but this will do the trick!:
function parseall(str) return Meta.parse("begin $str end").args end
As @Alexander Morley pointed out above,
parse is designed to only parse a single expression. So if you just make that single expression able to contain all the expressions in your original string, then it will parse it just fine! :)
The simplest way to do that is to wrap the string in a block, which makes it a valid julia expression. You can see that such an expression parses just fine:
julia> Meta.parse(""" begin function f3(x) x + 2 end print("N") print(f3(5)) end """) >> quote #= none:2 =# function f3(x) #= none:3 =# x + 2 end #= none:5 =# print("N") #= none:6 =# print(f3(5)) end
This is in fact the structure that the code in @Alexander Morley's is building. (And that's what made me think of doing it this way! Thanks Alexander!)
However, do note that if you are trying to parse a file, you actually don't want to return a single
block expression like that returns, because you can't eval it as a single block: Julia doesn't allow you to nest some "top-level statements" (Modules) inside of anything. Instead you want to return an array of top-level expressions. That's why we return the
args out of the Block expression.
With the above, you can
eval each of those top-level expressions, and it will work correctly! :)
julia> for expr in parseall("module M f() = 1 end"); Core.eval(Main, expr); end julia> M.f() 1
This part was figured out together with @misakawa (@thautwarm on Github).
parse is designed to parse a single expression (at least that's what the docs say: given this I'm actually a bit surprised your first example works , without throwing an error...).
If you want to parse mutliple expressions then you can take advantage of the fact that:
parsecan take a second argument
startthat tells it where to start parsing from.
- If you provide this start argument then it returns a tuple containing the expression, and where the expression finished.
to define a
parseall function yourself. There used to be one in base but I'm not sure there is anymore. Edit: there is still in the tests see below
# modified from the julia source ./test/parse.jl function parseall(str) pos = start(str) exs =  while !done(str, pos) ex, pos = parse(str, pos) # returns next starting point as well as expr ex.head == :toplevel ? append!(exs, ex.args) : push!(exs, ex) #see comments for info end if length(exs) == 0 throw(ParseError("end of input")) elseif length(exs) == 1 return exs else return Expr(:block, exs...) # convert the array of expressions # back to a single expression end end