Building a chatbot with Julia, recommend starting points

General Usage
1

Hello everyone,
I am new to programming and decided to pick Julia as my first programming language. My goal is to build an online chatbot which is aimed at interfacing with a specific website which doesn’t have an API and doesn’t support chatbot functionality of any kind (said platform is kind of antediluvian). So just to clarify - my browser is doing the interaction with said website.

The goal of this chatbot is to basically parse HTML inputs made by another person in a chat occurring on said website and make simple choices based on that input (e.g. either to reply in a certain way or to end the chat). Choices are made based on pre-defined criteria (somehow) set in the chatbot (e.g. if input is (any of x) do 1, if input is (any of y) do 2, else do 3. The way my chatbot could interface with that website is by interacting with various objects on that website (e.g. if x happens click on this red button aka element with this id or class). This will be sufficient for achieving something that I pompously call “stage 1 autonomy”. For stages 2 and 3 I’d like to research NLP and possibly integration with GPT-3 API, but those are things I don’t even understand so at this point they are irrelevant to my current goal.

So based on those requirements could you recommend me some starting points (e.g. what libs to take a look at, what books to read, what julia videos to watch etc.)? Also can you recommend me a strategy for tackling this problem (e.g. should I try old-school html parsing or should I use something like Selenium/Webdriver etc.). What techstack do I need for this? Ideally I’d like to build this project where I won’t have to do full rewrite when I start adding NLP functionality, but that might be impossible, so it’s not a priority.

Anyway thanks to all who provide help and suggestions.

2 Likes
2

Wow, okay. Absolutely no responses. I guess I picked the wrong programming language.

3

yeah, you might as well picked the wrong language or approach. You need Selenium and there’s WebDriver.jl, but at this point, it’s less about programming in Julia than looking at APIs of Selenium. Your question is really broad, it’s hard to start answering which is probably why you didn’t get responses yet.

It’s much easier to answer a question like:
“I’m trying to get Julia to read and send message to this website X (actual URL), I tried using HTTP.jl / blah blah, and it didn’t work, here’s the snippet that I think should work”

4

@jling I’m not sure what you meant but this doesn’t seem very welcoming.

@tatsumaru welcome! It’s a holiday weekend in the USA so people aren’t around. There are more people on The Julia Language Slack than on Discourse so you might try there if you aren’t getting much traction here.

However, I do think @jling has a point – for this project Python will provide a better experience with many more relevant resources. It has a much larger community for machine learning and especially web tech. The documentation for Julia’s web tools is very bare-bones, compared to very large documentation websites in Python.

In case you do want to use julia, there are two ways to interact with a website that doesn’t have an API: Use HTTP.jl to send requests to the website and read the responses or use Webdriver.jl to drive a browser. https://jonathandinu.com/blog/julia-web-scraping/ is a good web-scraping tutorial using HTTP.jl.

1 Like
5

We shouldn’t be shy about it when OP asks to do something, and using Julia is the roundabout way of doing it.

If someone asks how to write a small binary that does some lightweight stuff, idk, like taking and printing string, and they are new to programming, and they’re wondering if Julia is good. I won’t try to lead them in hours of PkgCompiler saga and 1.8 --strip-ir etc just because technically it’s possible.

Manipulating browser just doesn’t have much to do with Julia, it’s the same using anything, and I can guess python probably has more concrete tutorials and documentation but I could be wrong.

I don’t think it’s unwelcoming to be honest of what Julia is best and “meh” at, especially if the question comes from a new programmer.

6

OK here’s an HTML Parser

To respond to your snark

Wow, okay. Absolutely no responses. I guess I picked the wrong programming language.

I think tbh that yes, you did. Not because of the lack of responses but because you need a web stack.

Your chatbot is extremely ambitious. Especially for someone who doesn’t even know what language to use.

You’d be better off creating a chatbot without the Web in front of it. Perhaps IRC would be a place to start.

Good Luck.

7

Welcome to the Julia community, and Julia can be a great first programming language, I think. I do not speak from experience, as it was maybe my 10th, so I can’t confidently speak for first-user experience. I want to draw your attention to two books (I would take the ranking there with a grain of salt, some clearly do not apply to you (yet), I at least know those two books I’ve bought):

https://bookauthority.org/books/best-julia-programming-books

Tanmay Teaches Julia for Beginners: A Springboard to Machine Learning for All Ages

Written by 15-year-old technology phenom Tanmay Bakshi, the book is presented in an accessible style that makes learning easy and enjoyable. Tanmay Teaches Julia for Beginners: A Springboard to Machine Learning for All Ages clearly explains the basics of Julia programming and takes a look at cutting-edge machine learning applications. You will also discover how to interface your Julia apps with code written in Python.

Julia Programming Projects: Learn Julia 1.x by building apps for data analysis, visualization, machine learning, and the web

Unlike Tanmay, Adrian Salceanu the author of the second book, is a regular here on discourse, known for web programming with Julia, maker of Genie.jl and Stipple.jl.

It’s good to have goals, I see you have at least 2-3 problems to solve, that to me do not seem like beginner-level.

I wouldn’t read too much into no responses after one day.

I did start a masters in NLP, and for programming NLTK was used (and the (online) book): https://www.nltk.org/

I DID use Julia with NLTK library, because it’s easy to call from Julia to Python with PyCall.jl (and I always use Julia if I can, at least to see if fits the problem), and the teacher was ok with it. I wouldn’t say NLTK, or the book are outdated, I guess still good for learning, but I understand spaCy is now the go to library, in Python land: https://spacy.io/

I suppose you could, and should use spaCy, last time I checked it had an immature Julia wrapper, but you don’t strictly need a wrapper, you can just use PyCall.jl to use the Python API. That can apply to other libraries you use too. Then does Julia provide an advantage for you over just using just Python, which is the default language in NLP/NLU. That’s unclear. Long-term, Julia is better for that and AI in general, I would think, I’m hoping Julia will take over, but it’s unclear it will happen.

2 Likes