/ is the prioritized choice operator. If the first expression succeeds, the whole expression succeeds. If the first expression fails, it backtracks and evaluates the second expression.
! is the not predicate operator which succeeds if its operand fails. It never consumes any input.
. matches any input.
The grammar can be translated to boo very simply using the peg macro from Boo.Pegs:
I had to be a little creative in mapping the PEG operators to valid boo expressions because as it must be clear by now boo doesn't allow the introduction of completely new syntax and that's what the fuss is all about here.
I actually like the way it looks.
Implementing something more useful such as expression evaluation on top of that requires a few semantic actions operating a stack:
Semantic actions are just closures that get executed as matching succeeds. $text returns the text matched so far by the current rule.
Beautiful.
The underlying implementation based on a graph of expression objects really shines when one considers what it takes to extend the grammar above with support for hexadecimal literals:
It's not yet clear how this extensibility mechanism will be exposed at the boo language level but the simplicity at the peg level is encouraging.
One last feature worth pointing out is the ability to match based on a previously matched rule. For instance, the closing tag of a xml element must match the name in the starting tag:
import Boo.Pegs
peg:
element = '<', tag, '>', content, '', @tag, '>'
tag =++(a-z)
content =--(element / text)
text =not "<", any()assert element.Match(PegContext("<foo><bar>Hello</bar></foo>"))
I've found the idea for the lastmatch operator @ first described in this article. Great idea.
I've been also greatly inspired by conversations I've had with Massi who's exploring similar territory and Jb during the last Mono Meeting in Barcelona Madrid and with Cedric over a beer in Paris. I think Massi will be pleased to know that I haven't given any thoughts to performance leaving all the fun to him.
Extensible parsing. Soon in a boo compiler close to you.
It was more than twenty years ago that Dijkstra wrote against the perils of anthropomorphism on science. And here we stand building whole industries on top of it.
But is it programming dominantly math? Or is it mainly human communication?
Maybe we swung too far to the the latter. I speculate that the current rise of functional programming can also be attributed to its liberating effect - one is no longer expected to attribute proper intent to entities before a program can be seen to make sense.
As a constant reminder of that I'm tempted to use a different font for my programming.
"One of the things you'll discover as you learn more about macros is how much day-to-day coding in other languages consists of manually generating macro expansions."
Gaiaware has just announced the Gaia Programming Contest. A contest "... about creating an Ajax Application that will serve as a meeting place for people dedicated to solving environmental issues...".
Very good but there's more: "... no Close Source dependencies can be used which means that the end product must be compilable on Mono ...". Great!
Of course people using boo and db4o have a huge headstart. So what are you waiting for?
A chance to interact live with a dear friend. Free Software, Hacking, Women, Futurama, McDonalds, love spreading, Militant Atheism, Monty Python, Douglas Adams and the French Way.
Had lots of interesting exchange of ideas with Massi, ranging from "extensible parsing through composeable PEGs with optimal performance" to Carlos Castañeda, Jesus Christ, meta-physics, religion and Pink Floyd. The Cryptonomicon really got me.
Got to put a face on Joachim and see how really cool Unity is.
On Thursday I got to talk about db4o which led me to meet a few db4o users hanging around the conference.
Pedro Santos had an interesting question, how to monitor and control the usage of computational resources in a managed client/server application? In other words, how a sysadmin can make sure a specific client won't DOS the application?
For .NET servers running on Windows there are performance counters, what about Mono servers running on Linux?
I've also got to spread the gospel about boo for which I got a hugely positive response.
Thomas "Gaia" Hansen seemed to really get it and so we had lots of interesting discussions on how to take over the world boo style.
Jackson wants to hack on better nullable type support for boo!
Mark wants an extensible language where NullReferenceExceptions are impossible.
Miguel reassured me once again mcs won't be rewritten on top of the boo compiler infrastructure :)
I had the most fascinating dream last night, very complex and full of details and by its very nature impossible for me to describe it. It was mix of the Tao, The Hitchhiker's Guide to The Galaxy, Big Bang Theory and Computer Science and if I was to sum it all up it would be something like "the multiverse as a breadth-first search algorithm".
So what would be the optimal universe configuration? Looking from where I stand it's hard to believe it's going to be ours.
I don't believe I'm the first one to think it that way so now I'm googling for references. Drop me a line if you know of any.
Update 1:this seems to be close enough so I'm definitely getting it.
Update 2: I got depth-first and breadth-first mixed up :)
Oren talks about a simple but interesting macro to aid with mocking. I decided to see if and how the latest meta programming facilities I've been working on are actually useful. Here's the complete application, what do you think?
namespace Adapter
import Boo.Lang.Compiler
import Boo.Lang.Compiler.Ast
import Boo.Lang.Compiler.Ast.Visitors
import Boo.Lang.Compiler.TypeSystem
import Boo.Lang.Compiler.MetaProgramming
class AdapterMacro(AbstractAstMacro):def Expand(macro as MacroStatement):if macro.Arguments.Count != 1 ornot macro.Arguments[0]isa ReferenceExpression:raise "adapter must be called with a single argument"
entity = NameResolutionService.Resolve(macro.Arguments[0].ToString())raise "adapter only accept types" unless entity.EntityType == EntityType.Type
BuildType(macro, entity)def GetModule(node as Node)as Boo.Lang.Compiler.Ast.Module:return node.GetAncestor(NodeType.Module)def BuildType(macro as MacroStatement, type as IType):
adapterInterface =[|interface $("I" + type.Name):pass|]
adapter =[|class $(type.Name + "Adapter")($adapterInterface):
theTarget as $(type.FullName)defconstructor(target as $(type.FullName)):
theTarget = target
|]
GetModule(macro).Members.Add(adapter)
GetModule(macro).Members.Add(adapterInterface)for member in type.GetMembers():
AddMethod(adapter, adapterInterface, member)if member isa IMethod
BooPrinterVisitor(System.Console.Out).Visit(adapterInterface)
BooPrinterVisitor(System.Console.Out).Visit(adapter)def AddMethod(adapter as ClassDefinition,
adapterInterface as InterfaceDefinition,
method as IMethod):ifnot method.IsPublic:returnif method.IsStatic:returnif method.ReturnType.IsByRef:returnif method.ReturnType.IsArray:return
interfaceMethod =[|def $(method.Name)()as $(method.ReturnType.FullName):pass|]
forwarder = interfaceMethod.CloneNode()
forwardInvocation =[| theTarget.$(method.Name)()|]for param in method.GetParameters():if param.IsByRef or param.Type.IsArray:return
forwarder.Parameters.Add(
ParameterDeclaration(
Name: param.Name,
Type: SimpleTypeReference(param.Type.FullName)))
interfaceMethod.Parameters.Add(
ParameterDeclaration(
Name: param.Name,
Type: SimpleTypeReference(param.Type.FullName)))
forwardInvocation.Arguments.Add(ReferenceExpression(param.Name))
adapterInterface.Members.Add(interfaceMethod)
adapter.Members.Add(forwarder)if method.ReturnType == TypeSystemServices.VoidType:
forwarder.Body.Add(forwardInvocation)else:
forwarder.Body.Add([|return $forwardInvocation |])
code =[|import Adapter
adapter intprint Int32Adapter(42)isa IInt32
|]try:
module = compile(code, typeof(AdapterMacro).Assembly)
module.EntryPoint.Invoke(null, (null,))except x as CompilationErrorsException:print x.Errors.ToString(true)
Today history was made. The first eclipse plugin written in boo and compiled down to java bytecodes by boojay has come to life. The plugin is a direct translation of the the plugin described here.
And the boo code:
"""
Hello World eclipse plugin.
A direct translation of
http://www.eclipse.org/articles/Article-Your%20First%20Plug-in/YourFirstPlugin.html
to boo.
"""
namespace HelloWorldPlugin
import org.eclipse.ui
import org.eclipse.jface.action
import org.eclipse.jface.dialogs
import org.eclipse.jface.viewers
class HelloWorldAction(IWorkbenchWindowActionDelegate):
activeWindow as IWorkbenchWindow
def run(proxyAction as IAction):shell= activeWindow.getShell()
MessageDialog.openInformation(shell, "Hello from boojay!", "Hello World!")def init(window as IWorkbenchWindow):
activeWindow = window
def dispose():passdef selectionChanged(proxyAction as IAction, selection as ISelection):pass
A boo application using the SWT java GUI library. Thanks to IKVM that's not only possible but very simple as well.
So what's the news? Well, Friday morning I was chatting with Klaus and he said to me "if you get boo to emit java bytecodes I'll do all my stuff in boo". How's that for a challenge? :)
I've finally took some time off this weekend to implement a simple object pattern matching facility as part of the newly created boo-extensions project.
Here's some code using the new 'match' and 'data' macros to implement an expression evaluator:
import Boo.PatternMatching
import Boo.Adt
def eval(e as Expression)asint:
match e:
case Const(value):return value
case Add(left, right):return eval(left)+ eval(right)def simplify(e as Expression)as Expression:
match e:
case Add(left: Const(value: 0), right):return simplify(right)
case Add(left, right: Const(value: 0)):return simplify(left)
case Add(left, right):return Add(simplify(left), simplify(right))
case _:return _
data Expression = Const(value asint)| Add(left as Expression, right as Expression)
e = Add(Add(Const(19), Const(0)), Add(Const(0), Const(23)))print simplify(e)print eval(e)print eval(simplify(e))
'match' coupled with our recently acquired quasiquoting capabilities makes writing macros for boo a pleasant endeavour actually.
Dont believe me? Take a look at the data macro again.
It all really started when Bob Pasker asked me if db4o worked with scala. That kind of subtext is usually all I need to start exploring another programming language. Specially one of such a tasty functional flavor.
My experiment was to port a simple time tracking application I wrote for myself some time ago from .net/boo to jvm/scala.
The application works as a tray icon that lets you right-click your way through projects and tasks. There's no reporting interface other than a REPL window that allows you to execute arbitrary code against the app's object model :)
It actually took me only a week to get it up and running on the three major platforms I work with (windows, linux and macosx) thanks to scala, db4o and swt.
My impression so far is pretty darn good.
Scala is a beautiful language.
The tight integration with java means great tools such as db4o work out of the box.
And the eclipse support goes as far as supporting eclipse plugins written in scala (niiiice).
If you do java you should really be giving scala a ride.
By the way named arguments can also be used with meta methods:
a = dict(A: "foo", B: "bar")
Possible implementation:
[meta]def dict(keywords as(ExpressionPair)):
h =[|{}|]for pair in keywords:
h.Items.Add([| $((pair.First as ReferenceExpression).Name): $(pair.Second)|])return h
Keyword arguments are all collected into an array passed as the first argument to the meta method. That way, meta methods can still do pattern matching on the number and types of expression arguments.
Boo meta methods are methods that take code trees as input and return code trees as output. In addition they must be marked with the MetaAttribute for the compiler to recognize them as so.
The compiler invokes meta methods during the type resolution phase and replaces the code tree at the point of invocation with the code tree returned by the meta method.
Multiple overloads can be specified in which case the types of the code tree arguments will be used for the purpose of overload resolution.
An example, assert
Assert can be invoked with one or two arguments:
assert x is null // use the code as the assertion message
assert x isnot null, "x shouldn't be null" // custom exception or string
One way of implementing assert would be:
[meta]defassert(condition as Expression):return[|ifnot $condition:raise AssertionFailedException($(condition.ToCodeString()))|][meta]defassert(condition as Expression, exception as Expression):return[|ifnot $condition:raise $exception
|]
Where [| |] are the quasi-quotation delimiters. A quasi-quote evaluates its body as a code tree expression.
$ is generally called the "splice" operator and it means "evaluate me at compilation time".
Alternatively, assert could declare a variable parameter list:
The splice application $(condition.ToCodeString()) automatically lifts the string value returned by ToCodeString to a proper StringLiteralExpression.
Another example, using
'using' provides for deterministic disposal of resources. The argument should implement the IDisposable interface to have its Dispose method called
at the end of a provided code block.
For instance, the following code:
using socket=OpenConnection():
socket.Send("ping")
should be expanded to something equivalent to:
socket=OpenConnection()try:
socket.Send("ping")ensure:if socket isa IDisposable:(socket as IDisposable).Dispose()
A possible implementation for 'using' follows:
[meta]defusing(e as Expression, block as BlockExpression):
temp = uniqueName()return[|
$temp = $e
try:
$(block.Body)ensure:if $temp isa IDisposable:($temp as IDisposable).Dispose()|]
More about quasi-quotation
Due to the reuse of syntactic elements in different contexts the parser needs to follow some conventions in order to infer the meaning of a quasi-quote expression.
Take the quasi-quote [| a as string |]. What does it mean? If we were to interpret it as an expression it would mean a try cast expression. If we were to interpret it as a type member it would mean a field definition.
For the inline form the convention is to try to interpret it as either an expression or an expression pair or an import declaration or a namespace declaration.
The block form is first probed for a type member definition then for a single statement, then for a block of statements.and then for a module.
Example:
e =[| a asstring|] assert e isa TryCastExpression
f =[|
a asstring |] assert f isa Field
m =[| namespace Spam
print "Spam! "*3 |] assert m isa Module
It should be possible to specify the exact context of quasi-quotation by using some special syntax but the specific details are not clear at this point.
Soon to reach a svn repository near you.
Oh, yeah, and many thanks to the people behind Template Haskell!
With many thanks to the people who contributed for this release: Andrew Davey, Avishay Lavie, Cedric Vivier, Chris Prinos, Doug Holton, Jim Lewis and Max Bolingbroke.
It works pretty well except that all those do keywords and parenthesis kind of get in the way of clarity for that particular use case.
What we really would like to write is something like:
1:html:
2: body:
3: text "Hello, world!"
Which is clearer and more to the point.
Boo actually allows one to do such a thing by extending the language with macros.
Boo macros are objects that are invoked by the compiler during the compilation process to expand or transform the AST in a hopefully useful way.
Whenever the Boo parser finds code in the form: name expression_list (block)? it creates a MacroStatement node that will be later handled to a macro object for expansion.
Let's define our html, body and text macros for our DSL:
"Assuming that you have no access to tooling, and you don't have the resources to built NHibernate-sque framework, how would you approach building a Domain Driven application on the naked CLR?"
The most interesting part of the question for me is "and you don't have the resources to built NHibernate" because it immediately goes to the seemingly basic assumption most people have these days that "Persistence => SQL".
While it might be certainly true that a relational backend is a given for most enterprisey scenarios it is certainly not true that all persistent applications have to go through the pain.
Once upon a time a team with 4 people (2 developers, 2 web designers) built the web content management system for the 3rd largest TV station in Brazil on top of System.Runtime.Serialization using the Object Prevalence architecture.
Yeah, skin naked CLR.
If your specific application can't afford keeping all its objects in memory all the time and you don't mind putting a little clothes on, there's db4o.
Around 4 years ago (!) there was this discussion about how to support some dynamic language features on top of mono. One of the topics was optimizing dynamic dispatching and apparently my suggestion was redirected to nul.
4 years later here I am finally implementing the idea in order to take boo's dynamic dispatching performance to the next level.
Each line reports how long it takes to execute the described operation with dynamic dispatching 5_000_000 times (except the last line which executes the same operation as the line before it but with static dispatching).
The first line tells us that it takes 1.10 seconds for boo to multiply two integer objects using dynamic dispatching.
The second line says boo takes 29.02 seconds to multiply a List instance by an integer using dynamic dispatching (dynamic dispatch over static methods).
The third line which is the most interesting one for our purposes here says that boo takes roughly 52 seconds to dynamically dispatch 5_000_000 instance method calls.
We can see a huge overhead over static dispatching.
So this first stab got it from 52 seconds down to 4 seconds. Not bad at all. A few changes and we'll have the same benefits for dynamic dispatching over static methods.
I hope this will have a huge impact on environments that rely heavily on dynamic dispatching such as Brail.
Unfortunately though this optimization is only available when building for the .NET 2.0 profile.
I've just arrived in Seattle for the Microsoft DLR Compiler Lab. JB will be here soon and I heard Miguel is joining us as well. Fun!
Looking forward to getting my first Boo Silverlight application running.
One of the questions I'm here for is should we migrate Boo's duck typing support to be based on the DLR? On the pros side we get better integration with all the DLR languages and perhaps better performance when executing dynamic code. On the cons side it's an additional dependency (and not a very mature one for that matter).
I started reviewing the overload resolution code in the boo compiler this week. Very old and, let's say, very interesting code. It was based on a fuzzy scoring system I have no idea how I came up with. Well, these things happen.
The new code is based on the concepts discussed here. Thanks Avish and Daniel for that!
A few test cases (mostly varargs related) had to be reviewed to comply with the improved behavior so expect a few compilation errors when updating to the new code and please let us know of any strange behavior.
