JunOS config yacc grammar?
Any chance somebody out there has a yacc grammar that will parse a Juniper config files? My immediate interest involves v19.X on our EX4300s, but anything in the ballpark would save me having to write one from scratch. --lyndon
Lyndon Nerenberg (VE7TFX/VE6BBM) wrote on 21/08/2023 22:14:
Any chance somebody out there has a yacc grammar that will parse a Juniper config files? My immediate interest involves v19.X on our EX4300s, but anything in the ballpark would save me having to write one from scratch.
No need to reinvent that wheel: root@foo> show configuration | display xml root@foo> show configuration | display json ... then slurp into an ingestion engine in your favourite language. Nick
Nick Hilliard writes:
No need to reinvent that wheel:
root@foo> show configuration | display xml root@foo> show configuration | display json
That doesn't quite work for this scenario. It would mean ssh-ing to the switch to grab it, and that's pretty locked down. We already have cron jobs running on the switches that tftp the config file to a server, and I'd prefer to leverage off that. Also, the yacc parser would let me do some validation checks before we push new configs. --lyndon
I would first try to understand what you are trying to achieve. JUNOS is very flexible on this front and I am wondering why you think yacc is the right way to achieve what you are trying to do. If no one (or very few) these days is using yacc grammar for parsing router configs, that should be a good indication whether you are on the right path or not. As already pointed out, navigating the XML trees or JSON structures would be much easier than writing a yacc grammar parser. Alternatively, I would look into the YANG files: https://github.com/Juniper/yang But without understanding what you are trying to do, this is just another suggestion. ./diogo -montagner JNCIE-SP 0x41A On Tue, 22 Aug 2023 at 09:53, Lyndon Nerenberg (VE7TFX/VE6BBM) < lyndon@orthanc.ca> wrote:
Nick Hilliard writes:
No need to reinvent that wheel:
root@foo> show configuration | display xml root@foo> show configuration | display json
That doesn't quite work for this scenario. It would mean ssh-ing to the switch to grab it, and that's pretty locked down. We already have cron jobs running on the switches that tftp the config file to a server, and I'd prefer to leverage off that.
Also, the yacc parser would let me do some validation checks before we push new configs.
--lyndon
Diogo Montagner writes:
--000000000000728632060377d0b2 Content-Type: text/plain; charset="UTF-8"
I would first try to understand what you are trying to achieve. JUNOS is very flexible on this front and I am wondering why you think yacc is the right way to achieve what you are trying to do.
Because I've been writing yacc grammars for decades. I just wanted to see if someone had already done it, as that would save me some time. But if there's nothing out there I'll just roll one myself. --lyndon
On Tue, 22 Aug 2023 at 03:30, Lyndon Nerenberg (VE7TFX/VE6BBM) <lyndon@orthanc.ca> wrote:
Because I've been writing yacc grammars for decades. I just wanted to see if someone had already done it, as that would save me some time. But if there's nothing out there I'll just roll one myself.
I sympathise with your problem and I've always wanted vendors to publish their parsers, there are many use cases. But as such does not exist, this avenue of attack seems very problematic, unless this whole network lives and dies with you. If not, then your feature velocity now depends on someone adding support for new keywords to the parser generator, no one who comes after you will thank you for adding this dependency to the process. But they might call you and pay stupid money for a 5 min job, so maybe it is a great idea. -- ++ytti
Lyndon Nerenberg (VE7TFX/VE6BBM) wrote on 22/08/2023 01:27:
Because I've been writing yacc grammars for decades. I just wanted to see if someone had already done it, as that would save me some time. But if there's nothing out there I'll just roll one myself.
check out xorp and vyos - both contain code to parse junos style configurations. Just bear in mind that they provide basic tokeniser functionality, which parses the configurations into token trees. The config interpretation can then be handled on a modular basis. Nick
I agree with the yang approach. I would probably convert to using Yang and download the yang files for whatever code version you are running. For every code version you can download the yang files. Then you have a choice to decide, either right a yang to yacc conversion tool or find one, or think about switching to a netconf approach or ODL/restconf -> netconf style approach depending on how you are using the yacc files today as that will auto check and verify basics and reject non compliant configs to some level. On Tue, Aug 22, 2023 at 9:36 AM Diogo Montagner <diogo.montagner@gmail.com> wrote:
I would first try to understand what you are trying to achieve. JUNOS is very flexible on this front and I am wondering why you think yacc is the right way to achieve what you are trying to do.
If no one (or very few) these days is using yacc grammar for parsing router configs, that should be a good indication whether you are on the right path or not.
As already pointed out, navigating the XML trees or JSON structures would be much easier than writing a yacc grammar parser.
Alternatively, I would look into the YANG files: https://github.com/Juniper/yang
But without understanding what you are trying to do, this is just another suggestion.
./diogo -montagner JNCIE-SP 0x41A
On Tue, 22 Aug 2023 at 09:53, Lyndon Nerenberg (VE7TFX/VE6BBM) < lyndon@orthanc.ca> wrote:
Nick Hilliard writes:
No need to reinvent that wheel:
root@foo> show configuration | display xml root@foo> show configuration | display json
That doesn't quite work for this scenario. It would mean ssh-ing to the switch to grab it, and that's pretty locked down. We already have cron jobs running on the switches that tftp the config file to a server, and I'd prefer to leverage off that.
Also, the yacc parser would let me do some validation checks before we push new configs.
--lyndon
On 8/21/23 7:09 PM, Diogo Montagner wrote:
I would first try to understand what you are trying to achieve. JUNOS is very flexible on this front and I am wondering why you think yacc is the right way to achieve what you are trying to do.
Drive by comment: Perhaps the OP is trying to parse a (pile of) config file(s) downstream of the generation thereof and has no ability to alter their generation. Grant. . . .
On Tue, Aug 22, 2023 at 11:39 PM Grant Taylor via NANOG <nanog@nanog.org> wrote:
On 8/21/23 7:09 PM, Diogo Montagner wrote:
I would first try to understand what you are trying to achieve. JUNOS is very flexible on this front and I am wondering why you think yacc is the right way to achieve what you are trying to do.
Drive by comment:
Perhaps the OP is trying to parse a (pile of) config file(s) downstream of the generation thereof and has no ability to alter their generation.
this is a common problem (or is common when I look at things, perhaps I'm looking wrongly, but...) I'd love to have something that parsed all of my device type configs and output the results into a 'database' that i could then ask questions of like: "Hey, what NTP servers are configured on all devices?" "Hey, which devices have this <access-list/firewall/user> configured on them?" There are a host of other things I could ask those are but 2 simple examples, and YES I can grep/sed/awk|sort|uniq|sort-rn my way to success for the 2 examples I provided... but really that's NOT the way I want to do this, and I do really have a bunch of other questions I'd like to ask, regularly, to solve rollout-of-new-feature / compliance / legal / troubleshooting / etc questions. In looking around there are examples of some of this, in a way, the most common thing I end up looking at, and getting sad about, is some java monstrosity (who's name escapes me) but has shown up in a few nanog presentations over the years... it makes me sad because it's not super useful in my world :( 'hard to use' is probably the best way to describe it. One note about XML and Juniper, the schema changes by OS version, it changes quite a bit :( You CAN parse through it reasonably well with python lxml.Etree, because (I think) python's parse is VERY forgiving. If you attempt this path with golang :( you will be sad, very sad :( because the go->xml world is very 'build a struct of structs that mirrors the xml tree' and 'changes at every OS version' means now you have a LOT of versions of that :( maintenance gets back to saku's comment about feature velocity :( I do see: https://pypi.org/project/juniper-nxpy/ which may be useful to you as well Lyndon. (I'd also point to tftp as not being the super best option from a security and reliability perspective, but if that's what you've got that's what you've got... you COULD have the switch cronjobs curl/post to an https destination with little hard work, and a gain in reliabilty/security) -chris
Christopher Morrow wrote:
In looking around there are examples of some of this, in a way, the most common thing I end up looking at, and getting sad about, is some java monstrosity (who's name escapes me) but has shown up in a few nanog presentations over the years... it makes me sad because it's not super useful in my world :( 'hard to use' is probably the best way to describe it.
You're probably thinking of Batfish. Justin H.
On Thu, Aug 24, 2023 at 11:10 AM Justin H. <justindh.ml@gmail.com> wrote:
Christopher Morrow wrote:
In looking around there are examples of some of this, in a way, the most common thing I end up looking at, and getting sad about, is some java monstrosity (who's name escapes me)
You're probably thinking of Batfish.
yes! I kept wanting to say Bee... something :) thanks!
FWIW, I find the config archival on-commit to be a very useful feature. I use it with SCP. Sadly, neither J nor C support doing this with an SSH private key on the router, you have to use a password. J at least encrypts the password if you do it right (…”URL” password “plain-text”). Cisco does not encrypt this password in the config (perhaps they do in more recent releases. All my C hardware is ancient. C will do it on “write”. Not quite as good as “on-commit”, but since C lacks commit, is what it is. YMMV. Owen
On Aug 24, 2023, at 07:11, Christopher Morrow <morrowc.lists@gmail.com> wrote:
On Tue, Aug 22, 2023 at 11:39 PM Grant Taylor via NANOG <nanog@nanog.org> wrote:
On 8/21/23 7:09 PM, Diogo Montagner wrote: I would first try to understand what you are trying to achieve. JUNOS is very flexible on this front and I am wondering why you think yacc is the right way to achieve what you are trying to do.
Drive by comment:
Perhaps the OP is trying to parse a (pile of) config file(s) downstream of the generation thereof and has no ability to alter their generation.
this is a common problem (or is common when I look at things, perhaps I'm looking wrongly, but...) I'd love to have something that parsed all of my device type configs and output the results into a 'database' that i could then ask questions of like: "Hey, what NTP servers are configured on all devices?" "Hey, which devices have this <access-list/firewall/user> configured on them?"
There are a host of other things I could ask those are but 2 simple examples, and YES I can grep/sed/awk|sort|uniq|sort-rn my way to success for the 2 examples I provided... but really that's NOT the way I want to do this, and I do really have a bunch of other questions I'd like to ask, regularly, to solve rollout-of-new-feature / compliance / legal / troubleshooting / etc questions.
In looking around there are examples of some of this, in a way, the most common thing I end up looking at, and getting sad about, is some java monstrosity (who's name escapes me) but has shown up in a few nanog presentations over the years... it makes me sad because it's not super useful in my world :( 'hard to use' is probably the best way to describe it.
One note about XML and Juniper, the schema changes by OS version, it changes quite a bit :( You CAN parse through it reasonably well with python lxml.Etree, because (I think) python's parse is VERY forgiving. If you attempt this path with golang :( you will be sad, very sad :( because the go->xml world is very 'build a struct of structs that mirrors the xml tree' and 'changes at every OS version' means now you have a LOT of versions of that :( maintenance gets back to saku's comment about feature velocity :(
I do see: https://pypi.org/project/juniper-nxpy/
which may be useful to you as well Lyndon. (I'd also point to tftp as not being the super best option from a security and reliability perspective, but if that's what you've got that's what you've got... you COULD have the switch cronjobs curl/post to an https destination with little hard work, and a gain in reliabilty/security)
-chris
On Thu, Aug 24, 2023 at 10:10 AM, Christopher Morrow < morrowc.lists@gmail.com> wrote:
On Tue, Aug 22, 2023 at 11:39 PM Grant Taylor via NANOG <nanog@nanog.org> wrote:
On 8/21/23 7:09 PM, Diogo Montagner wrote:
I would first try to understand what you are trying to achieve. JUNOS is very flexible on this front and I am wondering why you think yacc is the right way to achieve what you are trying to do.
Drive by comment:
Perhaps the OP is trying to parse a (pile of) config file(s) downstream of the generation thereof and has no ability to alter their generation.
this is a common problem (or is common when I look at things, perhaps I'm looking wrongly, but...) I'd love to have something that parsed all of my device type configs and output the results into a 'database' that i could then ask questions of like: "Hey, what NTP servers are configured on all devices?" "Hey, which devices have this <access-list/firewall/user> configured on them?"
Isn't this YANG/NETCONF, and squish it all into DB/directory full of files? Basically a more standardized format for representing device configurations / states? W There are a host of other things I could ask those are but 2 simple
examples, and YES I can grep/sed/awk|sort|uniq|sort-rn my way to success for the 2 examples I provided... but really that's NOT the way I want to do this, and I do really have a bunch of other questions I'd like to ask, regularly, to solve rollout-of-new-feature / compliance / legal / troubleshooting / etc questions.
In looking around there are examples of some of this, in a way, the most common thing I end up looking at, and getting sad about, is some java monstrosity (who's name escapes me) but has shown up in a few nanog presentations over the years... it makes me sad because it's not super useful in my world :( 'hard to use' is probably the best way to describe it.
One note about XML and Juniper, the schema changes by OS version, it changes quite a bit :( You CAN parse through it reasonably well with python lxml.Etree, because (I think) python's parse is VERY forgiving. If you attempt this path with golang :( you will be sad, very sad :( because the go->xml world is very 'build a struct of structs that mirrors the xml tree' and 'changes at every OS version' means now you have a LOT of versions of that :( maintenance gets back to saku's comment about feature velocity :(
I do see: https://pypi.org/project/juniper-nxpy/
which may be useful to you as well Lyndon. (I'd also point to tftp as not being the super best option from a security and reliability perspective, but if that's what you've got that's what you've got... you COULD have the switch cronjobs curl/post to an https destination with little hard work, and a gain in reliabilty/security)
-chris
It may have been covered already, but another place to look is at code for Batfish https://github.com/batfish/batfish https://www.batfish.org/ Its goal. and there are even podcast episodes that cover it: https://www.podchaser.com/podcasts/episode-archive-packet-pushers-1755/episo... https://blog.ipspace.net/2019/09/intent-based-networking-with-batfish-on.htm... was to parse configs and out of that derive overall topology forwarding graph and do config verification, so the tool itself may be of some use, but for their parsing they use ANTLR, so you can reuse some of the code. Which would be a pretty useful functionality, because then you could build router stack independent policy processors and verification. And it would be nice to have RC9067 policy parser and compiler into proprietary config schema of one's choice. With some verification. Yan On Thu, Aug 24, 2023 at 1:04 PM Warren Kumari <warren@kumari.net> wrote:
On Thu, Aug 24, 2023 at 10:10 AM, Christopher Morrow < morrowc.lists@gmail.com> wrote:
On Tue, Aug 22, 2023 at 11:39 PM Grant Taylor via NANOG <nanog@nanog.org> wrote:
On 8/21/23 7:09 PM, Diogo Montagner wrote:
I would first try to understand what you are trying to achieve. JUNOS is very flexible on this front and I am wondering why you think yacc is the right way to achieve what you are trying to do.
Drive by comment:
Perhaps the OP is trying to parse a (pile of) config file(s) downstream of the generation thereof and has no ability to alter their generation.
this is a common problem (or is common when I look at things, perhaps I'm looking wrongly, but...) I'd love to have something that parsed all of my device type configs and output the results into a 'database' that i could then ask questions of like: "Hey, what NTP servers are configured on all devices?" "Hey, which devices have this <access-list/firewall/user> configured on them?"
Isn't this YANG/NETCONF, and squish it all into DB/directory full of files?
Basically a more standardized format for representing device configurations / states?
W
There are a host of other things I could ask those are but 2 simple
examples, and YES I can grep/sed/awk|sort|uniq|sort-rn my way to success for the 2 examples I provided... but really that's NOT the way I want to do this, and I do really have a bunch of other questions I'd like to ask, regularly, to solve rollout-of-new-feature / compliance / legal / troubleshooting / etc questions.
In looking around there are examples of some of this, in a way, the most common thing I end up looking at, and getting sad about, is some java monstrosity (who's name escapes me) but has shown up in a few nanog presentations over the years... it makes me sad because it's not super useful in my world :( 'hard to use' is probably the best way to describe it.
One note about XML and Juniper, the schema changes by OS version, it changes quite a bit :( You CAN parse through it reasonably well with python lxml.Etree, because (I think) python's parse is VERY forgiving. If you attempt this path with golang :( you will be sad, very sad :( because the go->xml world is very 'build a struct of structs that mirrors the xml tree' and 'changes at every OS version' means now you have a LOT of versions of that :( maintenance gets back to saku's comment about feature velocity :(
I do see: https://pypi.org/project/juniper-nxpy/
which may be useful to you as well Lyndon. (I'd also point to tftp as not being the super best option from a security and reliability perspective, but if that's what you've got that's what you've got... you COULD have the switch cronjobs curl/post to an https destination with little hard work, and a gain in reliabilty/security)
-chris
On Thu, Aug 24, 2023 at 11:45 AM Warren Kumari <warren@kumari.net> wrote:
from: me this is a common problem (or is common when I look at things, perhaps I'm looking wrongly, but...) I'd love to have something that parsed all of my device type configs and output the results into a 'database' that i could then ask questions of like: "Hey, what NTP servers are configured on all devices?" "Hey, which devices have this <access-list/firewall/user> configured on them?"
Isn't this YANG/NETCONF, and squish it all into DB/directory full of files?
yup, sort of... I think the yang parts are still not 100% there, and also then I'd need to do the 1 thing I've been avoiding (aside from printing from a unix machine) so far: "learn yang things" :)
Basically a more standardized format for representing device configurations / states?
yup! really any way that would be relatively steady over time to go from: "router config on router over there ->" to: "now in some useful way stored in a 'database'" was/is my goal... or one goal of the many goals I attempt to get to a solution for every once in a while :)
We already have cron jobs running on the switches that tftp the config file to a server, and I'd prefer to leverage off that.
Use the same cron jobs to save a copy as xml/json and pull the file off. On Mon, Aug 21, 2023 at 7:52 PM Lyndon Nerenberg (VE7TFX/VE6BBM) < lyndon@orthanc.ca> wrote:
Nick Hilliard writes:
No need to reinvent that wheel:
root@foo> show configuration | display xml root@foo> show configuration | display json
That doesn't quite work for this scenario. It would mean ssh-ing to the switch to grab it, and that's pretty locked down. We already have cron jobs running on the switches that tftp the config file to a server, and I'd prefer to leverage off that.
Also, the yacc parser would let me do some validation checks before we push new configs.
--lyndon
Sorry, I don't know the answer to your initial inquiry | We already have cron jobs running on the switches that tftp the config file to a server, and I'd prefer to leverage off that. Perhaps an easy interim adaptation would be to modify your scheduled job to tftp the latest xml or json then clean it up? root@BB6> ...on | display xml | save /config/junos-config-latest.xml Wrote 139 lines of output to '/config/junos-config-latest.xml' root@BB6> ...n | save /config/junos-config-latest.json Wrote 169 lines of output to '/config/junos-config-latest.json' root@BB6> root@BB6> file list /config/ | grep latest junos-config-latest.json junos-config-latest.xml root@BB6> Hope you find the final solution, as the parser checks sound worth the squeeze.
participants (13)
-
Christopher Morrow -
Diogo Montagner -
Grant Taylor -
Joe Snyder -
Justin H. -
Lyndon Nerenberg (VE7TFX/VE6BBM) -
Mark Prosser -
Nick Hilliard -
Owen DeLong -
Saku Ytti -
Tom Beecher -
Warren Kumari -
Yan Filyurin