diff options
Diffstat (limited to 'swigweb/future.ht')
| -rw-r--r-- | swigweb/future.ht | 367 |
1 files changed, 0 insertions, 367 deletions
diff --git a/swigweb/future.ht b/swigweb/future.ht deleted file mode 100644 index f5ae5ea90..000000000 --- a/swigweb/future.ht +++ /dev/null @@ -1,367 +0,0 @@ -SWIG-1.3.12, SWIG-2.0 and Beyond - -<h2>SWIG-1.3.12, SWIG 2.0, and Beyond</h2> - -<p> -Author: David Beazley (beazley@cs.uchicago.edu) - -<p> -June 2, 2002 - -<p> -With the release of SWIG-1.3.12, I thought I'd take a few moments of -everyone's time to talk about the past, the present, and the future of -SWIG development. I'm really quite excited about the current release -because I think it represents a huge turning point in SWIG's -development. Furthermore, it is only the beginning of bigger and -better things to come. However, we definitely need your help. - -<p> -To put a little perspective on the discussion, I'd start with a few -development statistics. In the last 12 months, there have been over -300 entries added to the CHANGES log and over 4000 CVS commits. -Although that may not sound like a lot compared to a huge software -project, it is significant in the context of SWIG. As a point of -comparison, there has been more SWIG development this year than in any -other year of the project and more than in the previous three years -combined. This even includes the first few years of development in -which there was also a lot of activity. Furthermore, many of the -recent changes have been extremely non-trivial (e.g., templates, -namespaces, type system, operators, etc.). As a result, SWIG is more -capable than I ever imagined possible. - -<p> -Regrettably, I must admit that I've been a little negligent in -discussing the roadmap for where I thought this flurry of SWIG -development was actually headed. In part, this is because I've been -buried in work. However, the real reason is that I didn't really know -where we were going---except that in a time far far far away in the -future, we might arrive at some kind of "stable" release with a -version number other than "1.3.x". Needless to say, that's not a very -compelling story. -<p> -That said, I've spent a lot of time thinking about SWIG and trying to -wrap my brain around it. Specifically, just what is (or should be) -the primary focus of this project and what are we really trying to do? -That's what the rest of this message is about. - -<h3>SWIG Prehistory</h3> - -<p> -The first version of SWIG was written in 1995. The original system -was developed to help with some software wrapping problems I -encountered while writing molecular dynamics software at Los -Alamos. Later that year, I became interested in extending the wrapper -generator to support other scripting languages so it was rewritten in -C++ and modified with multiple backends (Tcl, Perl, and Guile). This -led to a first public release in February, 1996. Feedback from this -release led to a series of enhancements and the release of SWIG 1.0 in -September 1996. Throughout this process, my intent was to create a -tool that I would want to use---I never viewed the project as an -CS experiment in programming languages or software engineering. - -<h3>SWIG 1.1</h3> - -SWIG-1.1 (June, 1997) represented a series of enhancements that were -added in response to feedback at conferences and from users. Shadow -classes, exception handling, typemaps, and a number of more useful -features were added. However, the overall structure of the system was -relatively unchanged from the initial version. Following the release -of 1.1, a series of minor patch releases were made. This resulted in -the release of SWIG-1.1p5 in February, 1998. Unfortunately, this -release would remain the last "stable" release for quite a long -time---in fact, it is still listed as the last "stable" release on the -SWIG web page! - -<h3>SWIG Hell</h3> - -Even during the development of SWIG-1.1, it was clear that the whole -design of the system was deeply flawed. The implementation was a mess -and the C/C++ support was full of holes and nasty corner cases. -Furthermore, there was no unifying principle that tied all of the -different SWIG directives together. Not only that, fixing these -problems appeared to be nothing short of impossible---requiring a -total ground-up rewrite at best. The only redeeming quality was that -the system basically worked "well enough," it was extensively -documented, and its flaws mostly known. People could use it and there -were work-arounds for most common problems. - -<p> -To deal with the design problem, there were at least four attempts to -completely rewrite SWIG, some of which were attempted in parallel with -the work on SWIG-1.1. Unfortunately, none of these were ever -completed. The primary problem was a strong "second system" effect and -a desire to make SWIG do everything that one might conceivably want to -do with a wrapper generator (somehow). Clearly, this was a recipe for -disaster. In fact, all such attempts to rewrite SWIG were eventually -abandoned several years ago. In hindsight, I think the real problem -was that these rewrite efforts focused far too much attention on -implementation technique rather than principles. In short, the -failure of these efforts was due to a lack of clarity in understanding -how SWIG ought to work (regardless of how it was actually -implemented). - -<h3>SWIG Restart (1.3a1-1.3a5)</h3> - -<p> -Having languished for several years, the SWIG1.1p5 release had a -growing pile of maintenance issues. It didn't work for newer versions -of certain language modules and a lot of minor bug reports and feature -requests had been building up. With a lot of help from Loic Dachary and -Thien-Thi Nguyen, we put together the 1.3a1 release (February, -2000). This was mostly a bug fix release to 1.1p5 except that the -preprocessor module from SWIG1.2 was added and a lot of minor -enhancements were added. - -<p> -For the next six months, a massive effort was made to rewrite all of -SWIG's internal data structures (strings, lists, hashes, etc.). This -work was all going on underneath the covers while we tried to keep -SWIG in an operational state. The primary focus of this work was -really one of cleanup. Having given up on a total rewrite, perhaps -we could settle with making the implementation incrementally better -than before. In addition this, Matthias Koppe jumped on board to -reimplement the Guile module and to make other improvements to the system. - -<p> -An important aspect of these releases was that many parts of the -system not directly related to wrapper code generation were removed. -This included the documentation system and Objective-C support. These -were not removed because they weren't useful. Instead, the -documentation system was removed because it accounted for nearly half -of the special SWIG directives, yet had no direct bearing on what SWIG -actually did. Obective-C support was removed because it was tangled -with C++ support in a way that was very difficult to understand and -maintain. The removal of these features was seen as a way to vastly -simplify cleanup--and to buy some time where we could rethink their -role in a future release. - -<h3>SWIG Redux (1.3.6-1.3.11)</h3> - -This work, started in February 2001, is the beginning of the current -SWIG implementation. With the help of William Fulton, Matthias Koppe, -Lyle Johnson, Luigi Ballabio, Jason Stewart, Richard Palmer, Sam -Liddicot, and others, this work can best be described as the wholesale -destruction of everything remaining from SWIG-1.1. The language -module API, type system, the parser, numerous SWIG directives, and -SWIG library files---all destroyed or rewritten. Not only that, we -started introducing significant incompatibilities with -SWIG-1.1---mostly in an effort to correct past wrongs and get -ourselves out of the tangled mess of earlier versions. A huge number -of long-standing bugs and difficult feature requests have also been -resolved. - -<p> -The general goal of this development could best be described as an -attempt to reduce SWIG to an easily described set of general "ideas" -about how it should operate. Special SWIG directives have been -eliminated or combined with others. Different parts of the system have -been better integrated. Features not directly related to wrapper code -generation have been removed and the system has become more -focused. Changes in internal data structures and APIs have allowed -SWIG to capture more information from interface files and to resolve -hard corner cases. More often than not, these are things that you -never notice unless you are an old user and you suddenly realize that -a problem you had several years back has disappeared. - -<p> -Along with the destruction of old code, this work has quietly -introduced a new core--the most significant features of which are a -new C++ type system and multi-pass compilation. More importantly, -this work has really tried to provide a more principled foundation for -future SWIG development. However, just what is this "more principled -foundation?" - -<h3>Convergence (1.3.12)</h3> - -With the upcoming 1.3.12 release, SWIG is about to take a big leap -forward. Almost all of this is due to one realization---that almost -every hard problem in past SWIG releases has been directly related to -errors and limitations in its type system. Types are the key to -understanding the structure of C and C++ code. They are at the heart -of understanding advanced language features like namespaces, nested -classes, and templates. They are directly related to the data -marshalling that occurs in wrappers. Not only that, they interact -with nearly every SWIG directive. A proper type system *is* the -necessary foundation for moving SWIG forward. - -<p> -To be honest, I don't think that the emphasis on types is entirely -obvious. In fact, a great deal of work in past SWIG rewrites has -focused on the problem of C++ parsing. For instance, modifying the -parser to handle more advanced C++ code or representing parse trees as -XML. Furthermore, if one looks at the SWIG mailing list, you can find -a *lot* of messages related to issues of C++ parsing whereas almost no -one ever talks about types (well, other than typemaps). Even other -wrapper generation tools seems to spend a lot of time dealing with the -parsing issue. Although parsing is clearly important, I don't think it -has ever been the real problem in SWIG. This is because even though a -parser can tell you what's in a header file, it doesn't tell you -anything about how the different pieces of the system behave or how -they might interact. To do that, you need to do a lot more than just -parsing--and that's really the whole point. - -<p> -Although earlier 1.3 releases have made big improvements to the type -system, SWIG-1.3.12 is the first release that really tries to tackle -the type-system issue in a major way. We have patched nearly all -remaining holes in the type system and we have added full support for -C++ namespaces. Not only that, we have completely reimplemented C++ -template support in a way that supports templates, member templates, -and template specialization. Luigi and I are currently using this to -work on proper SWIG library support for parts of the C++ standard -library and the Standard Template Library (STL). Although some crusty -C programmers (present company excepted), might balk at such apparent -insanity, this work has impacted all parts of SWIG at all levels. -Even a variety of subtle errors in C support have been fixed by this -work. - -<p> -In addition to the type system work, SWIG-1.3.12 contains continued -reduction in the implementation. Directives have been removed, -refined, renamed, or consolidated. We're still narrowing the focus of -the system and working towards some kind of convergence. "Convergence -to what?!?", you ask. - -<h3>So, what is SWIG?</h3> - -In a nutshell, SWIG is a C/C++ declaration compiler that generates -wrapper code (okay, so you knew that much). However, to really -understand what SWIG is doing and where SWIG-1.3.x is headed, it is -useful to know that the whole system is essentially being built around -three extensions to the C++ type system: - -<ul> - -<li><p><b>Typemaps.</b> Typemaps are rules that define the process by which - data is converted between languages. They are fully integrated - with the C++ type system and they are applied using a type-based - pattern matching mechanism. All data conversion SWIG is - defined by typemaps and is fully reconfigurable. - -<li><p><b>Declaration annotation.</b> There are special directives that modify - the wrapping behavior of individual declarations. Declarations - can be selectively identified and decorated with arbitrary - attributes that affect wrapper generation. Like typemaps, - declaration matching is fully integrated with the C++ type system. - Almost all SWIG customization directives are a form of declaration - annotation. - -<li><p><b>Class extension.</b> The ability to extend classes and structures - with new methods/attributes when building wrappers. Classes - are part of the type system so class extension is naturally - integrated with the C++ type system as well (big surprise). - -</ul> -<p> -And that's it--this is the end-game of SWIG-1.3.x development. When -stabilized and documented it will become SWIG-2.0. - -<h3>The Bigger Picture</h3> - -I really want to emphasize that all of this work is part of a much -bigger picture. SWIG is used by a surprising number of people in -industry, government, and academia. It's used to build systems used -by millions of people every day. It has even shown up in video games -and other unlikely settings. Although SWIG doesn't have the same -visibility as many large software projects, over 12000 people have -downloaded SWIG-1.3.11 in the last 4 months. Clearly someone is using -it for something! Because of this, I think it is important for us to -work on moving SWIG towards a more solid foundation. Doing so will -give the system more credibility and long term viability---and it will -be a lot more fun to use! - -<p> -It's also worth noting that there are some rather interesting CS -connections at work here. Extensions to the type system and typemaps -have some interesting relations to work in programming languages. The -SWIG declaration annotation system and class extension feature seem -oddly similar to work in the emerging area of Aspect Oriented -Programming (AOP). There are undoubtedly connections to other areas -of software engineering and architecture. - -<p> -The key point is that SWIG isn't going to connect to anything if -no-one can quite describe what it is or how it works. - -<h3>SWIG-2.0 and the Future</h3> - -SWIG-1.3.12 still represents work in progress. There are bugs, the -documentation is still incomplete, and there are parts of the -implementation that are rather ugly. We are also still working out a -few very hard problems like nested classes, callback functions, and -overloading. A few old features are still missing (Objective-C, -documentation). However, I believe the end of the 1.3.x series is -near and achievable. - -<p> -Over the summer, a few more 1.3.x releases may appear but the current -plan is to aim for a SWIG-2.0 release in September. This release is -really moving towards the design principles described above and will -be a very major enhancement over SWIG-1.1. - -<p> -As for the future, a number of interesting ideas are on the table. I -want to add support for contracts/assertions in order to solve some -reliability issues that arise when retrofitting legacy codes with a -scripting interface. Support for an extension language has been -promoted by David Fletcher and was suggested by someone else on the -mailing list rather recently. I have a graduate student working on -SWIG support for the Microsoft CLR and .NET languages. Other work -might include support for alternative parsers, dynamically loadable -language modules, and so forth. However, none of this is really going -to materialize if we can't get the 2.0 release stablized. In fact, I -see SWIG-2.0 as a necessary step for moving forward with these ideas. - -<h3>We need your help! Yes, you.</h3> - -Nobody gets paid to work on SWIG. The developers are volunteers who -work in their spare time. Furthermore, SWIG is not supported by -investors, a large corporation, or research grants. I work on it -because it's fun, challenging, and useful. I presume that other -developers do the same. However, we only have limited resources and -we need your help. - -<ul> -<li><p> -If you like SWIG and find it useful, we need you to try new versions. - We want you to torture test the releases and to break them. We need - bug reports. No bug is too obscure or unimportant---we *will* fix it - if we can. We also need feedback about things that are annoying or - compatibility features that might help in going from 1.1 to 2.0. - -<li><p> We need help with documentation, examples, testing, libraries, and all - sorts of other aspects of the release. Even if you have never - written a SWIG language module or dived into its implementation, - there are many ways that you can help. Consider writing a case study - about how you wrapped a big library. Contribute tests that break the - implementation in horrible ways. Help with the web page or FAQ. - -<li><p> Most of the SWIG-1.3.x work has focused on the SWIG core. However, as - the 2.0 release nears, we will be working on a variety of enhancements - to the language modules. If there are things you would like to see - in any of the language modules, you need to let us know. - -<li><p> There are SWIG language modules that have not made it into the - distribution. Examples that I know about include ITCL, Swig-Eiffel, - and Swig-Lua. We would gladly make these part of the standard SWIG - distribution. However, we also need help to do it. Porting from - SWIG-1.1 is no easy task, but we're more than willing to help. It's - not as bad as one might imagine. - -<li><p> We are always looking for developers. Subscribe to - to the <a href="mail.html">swig-devel</a> mailing list - or send me email to get involved. -</ul> - -<h3>Acknowledgements</h3> - -I'd just like to thank everyone who has submitted feedback, bugs, made -contributions, and put up with my occasional thrashing over the years. -I welcome any comments about this document and how we can make SWIG even -better. - -<p> -Dave Beazley (beazley@cs.uchicago.edu) <br> -June 2, 2002 - |