* Xwidgets This is an experimental branch to enable embedding of GTK widgets inside an Emacs window. The Emacs abstraction is called an Xwidget, for eXternal widget, and also in reference to the Xembed protocoll. There is a demo file called xwidget-test.el which shows some of the possibilities. There are some screnshots at the emacswiki. Currently its possible to insert buttons, sliders, xembed widgets, and webkit in the buffer. It works similar to the support for images in Emacs. Adding more types of widgets should be fairly straightforward, but will require adapter code for each type. A difference from images is that xwidgets live their own life. You create them with an api, get a reference, and tie them to a particular buffer with a display spec. Each xwidget can have several views. In MVC terms, an xwidget is the model, and an xwidget-view is a view of the xwidget in a particular Emacs window. The xwidget code attempts to keep the visual appearance of the views in sync with through an Observer pattern implementation. This is necessary to support the Emacs window paradigm. ** building bzr co bzr+ssh://bzr.savannah.gnu.org/emacs/xwidget/ #the below compiler flags shouldn't be strictly necessary export CFLAGS=" -g -DGLYPH_DEBUG=1" ./configure --with-xwidgets --enable-asserts --with-x-toolkit=gtk3 make -j4 gdb -ex run src/emacs ** try it out If you have GTK3 and gtk-webkit installed, you should be able to start the embedded webkit browser now: M-X xwidget-webkit-browse-url If that didnt work out try the minimal demonstration instead: (load-library "xwidget-test") (xwidget-demo-a-button) It looks unimpressive, but it's a gtk button inside an Emacs buffer! *** webkit hints If you got webkit working, great! Please note, though, that the current support is far from a full fledged browser. My focus is on delivering a component that can be used to build a full emacs based browser on. Since I implement a browse-url function you can get quite far just by: (setq browse-url-browser-function 'xwidget-webkit-browse-url) then all Emacs browser interface systems work to a degree. heres stuff I use currenly - m-x anything-surfraw interfaces to search engines - C-o in org mode opens links inside org - m-x ffap opens links anywhere in a buffer - m-x gtk-lookup-symbol searches gtk docs - m-x bookmark-jump etc.. I'll add more examples as I go along. However theres lots of support missing, see TODO list for information. Briefly: - download handling - keyboard field navigation - isearch - history - sites that use flash. I dont really care about this issue so its unlikely to be fixed. Just a heads up. ** Stability Beginning with Summer 2011 I am now able to use Xwidget Emacs as my primary Emacs. That is good for the project and the stability of the code. At the time of writing I have 24 hour Emacs uptime with several embedded webkit browsers, Gnus, org-mode, tramp, etc. etc. That said, there are still many improvements that needs to be done, particularily in memory management. Expect xwidget emacs to leak heavily for now. ** timeline for inclusion in trunk The Emacs 24 feature freeze is passed, so xwidgets won't probably be merged until Emacs 25. OTOH since I now use xwidget emacs as my primary emacs, I will merge from trunk much more often than in the past. ** Thanks emacs-devel@gnu.org. There are very helpful people there. When I started the xwidget project I had no clue about the Emacs internals. * Brief overview of how xwidgets work Xwidgets work in one way like images in Emacs. You bind a display spec very similar to an image display spec to buffer contents. The display engine will notice the display spec and try to display the xwidget there. The display engine prepares space at the right place for the xwidget and so on for free, as long as we provide proper sizes and so on back to the redisplay engine. ** Issues The problem is that Emacs cant actually draw the widgets, as it can with images. Emacs must notify GTK about where the widgets should be, and how they should be clipped and so on, and this information must be given to GTK synchonous with Emacs display changes. Ok, so why is that difficult then? - How do we know when a widget is NOT to be drawn? The only way I found so far is having a flag for each xwdiget, that is reset before a redisplay. When an xwidget is encountered during display, the flag is set. After redisplay, iterate all xwidgets and hide those which hasnt been displayed. - The gtk socket type for embedding external applications is desirable but presents a lot of difficulties of its own. One difficulty is deciding which input events to forward, and when and how to do it. ** placement and clipping the entire emacs frame is a gtk window. we use the fixed layout manager to place xwidgets on the frame. coordinates are supplied by the emacs display engine. widgets are placed inside an intermediate window, called the widgetwindow. the widgetwindows are placed on the emacs frame. this way was chosen to simplify clipping of the widgets against emacs window borders. ** different strategies Integrating toolkit widgets(gtk in this case) and the emacs display engine is more difficult than your plain average gui application, and different strategies has been tested and will continue to be tested. There was a distinction between live xwidgets and phantom xwidgets, previous to the change to MVC. - the first aproach was to have the live xwidget on-screen, and move them about. the phantoms were generated by snapshoting the live xwidget. the drawback of that aproach was that the gtk toolkit is admirably lazy and doesnt draw the widget if its not actualy shown, meaning that the snapshots for the phantoms will show garbage. - the second aproach was to use composition support. that tells gtk that the widget should be drawn in an off-screen buffer and drawn on screen by the application. this has the primary advantage that the snapshot is always available, and enables the possibility of more eye-candy like drawing live and phantom widgets in different colors. the drawback is that its our own responsibility to handle drawing, which puts more of the display optimization burden on us. this is aproach worked so-so. - another aproach is to have both live and phantom widgets drawn on-screen by proxy gtk objects. the live xwidget will be entirely handled in an off-screen window, and the proxy objects will redirect events there. - combine on-screen and off-screen aproaches. maybe composition is the way to go for most cases, but on-screen xembeding is the way to go for particular special cases, like showing video in a window. off-screen rendering and whatnot, is not efficient in that particular case, and the user will simply have to accept that the phantom of a video widget isnt particularily beautiful. - The current and seemingly sanest aproach implements a MVC pattern. ** Testing ;;test like: ;; cd /path/to/xwidgets-emacs-dir ;; make all&& src/emacs -q --eval "(progn (load \"`pwd`/lisp/xwidget-test.el\") (xwidget-demo-basic))" ** MVC and Xembedd The MVC approach appears to be at least in principle robust for plain gtk widgets. For the interesting case of gtk sockets which implements an xembed host widget that allows for embedding other applications inside an Emacs window, the story gets more complex. The problem is that xembed is designed to plug an application window inside a a secket and thats it. You can't move a plug between sockets. I tried numerous hacks to get around this but there is nothing that works realy well. Therefore the Emacs part of the code will only expose well-defined interfaces. cooperating applications will be able to use the interface in a well defined manner. The problem is that there is no known xembeddable application that implement the needed type of functionality, which is allowing for creating new windows on the fly that plug into new sockets. Therefore I will attempt to provide an external application that wraps another application and through hacks attempts to provide the needed multi view xembed function. That way Emacs is sane and the insanity contained. This app will work by providing a socket that an app plugs into. The socket window is copied efficientlp by means of composition to a number of other windows, that then are plugged into the different Emacs sockets. * ToDo:s ** DONE allow xwidgets to report their size CLOSED: [2011-07-19 Tue 14:26] now we just hard code sizes. but webkit widgets for instance can report sizes that suit the content. support that. ** TODO BUG xwidget view ghosts - xwidget-webkit-browse-url somewhere - split window. now theres 2 webkit views - c-x 1 now theres 2 views but one is a ghost! one should have been deleted when its window died but that didnt work for some reason here. - m-x xwidget-cleanup the ghost goes away because we killed explicitly but this is just a workaround. xwidget_view_delete_all_in_window(w); in delete-window-internal is not sufficient. delete-other-windows-internal delete_all_subwindows unshow_buffer ** DONE BUG annoying backtrace CLOSED: [2011-07-19 Tue 14:28] (this no longer seems to happen even under heavy usage. seems merging from trunk helped. lots were happening in redisplay at this time in trunk.) sadly happens a lot. - happens even with no initialized xwidgets - + row->glyphs[area][i].face_id or similar code, so row is invalid for some reason. xwidgets currently disable some redisplay opimizations so it might be an actual emacs bug manifesting without optimizations. *** bt 1 /* Compute the width of this line. */ row->pixel_width = row->x; for (i = 0; i < row->used[TEXT_AREA]; ++i) row->pixel_width += row->glyphs[TEXT_AREA][i].pixel_width; (gdb) bt #0 0x000000000045c340 in compute_line_metrics (it=0x7fffffff8a20) at xdisp.c:17549 #1 0x00000000004603da in display_line (it=0x7fffffff8a20) at xdisp.c:18792 #2 0x0000000000457646 in try_window (window=23403045, pos=..., flags=1) at xdisp.c:15399 #3 0x00000000004559c9 in redisplay_window (window=23403045, just_this_one_p=0) at xdisp.c:14944 #4 0x0000000000450247 in redisplay_window_0 (window=23403045) at xdisp.c:13152 #5 0x00000000005fdcd9 in internal_condition_case_1 (bfun= 0x450208 , arg=23403045, handlers=12691046, hfun= 0x4501d9 ) at eval.c:1538 #6 0x00000000004501ba in redisplay_windows (window=23403045) at xdisp.c:13132 #7 0x000000000044f19c in redisplay_internal () at xdisp.c:12706 #8 0x000000000044f9f2 in redisplay_preserve_echo_area (from_where=7) at xdisp.c:12964 #9 0x0000000000568525 in swallow_events (do_display=1) at keyboard.c:4197 #10 0x0000000000422554 in sit_for (timeout=40, reading=1, do_display=1) at dispnew.c:5963 #11 0x000000000056512c in read_char (commandflag=1, nmaps=8, maps= 0x7fffffffd3f0, prev_event=12720514, used_mouse_menu=0x7fffffffd604, end_time=0x0) at keyboard.c:2689 #12 0x0000000000572c59 in read_key_sequence (keybuf=0x7fffffffd850, bufsize= 30, prompt=12720514, dont_downcase_last=0, can_return_switch_frame=1, ---Type to continue, or q to quit--- fix_current_buffer=1) at keyboard.c:9291 #13 0x0000000000562897 in command_loop_1 () at keyboard.c:1446 #14 0x00000000005fdb52 in internal_condition_case (bfun= 0x5624b4 , handlers=12772898, hfun=0x561dab ) at eval.c:1493 #15 0x00000000005621ab in command_loop_2 (ignore=12720514) at keyboard.c:1157 #16 0x00000000005fd4ce in internal_catch (tag=12768770, func= 0x562185 , arg=12720514) at eval.c:1247 #17 0x000000000056215e in command_loop () at keyboard.c:1136 #18 0x00000000005618f9 in recursive_edit_1 () at keyboard.c:757 #19 0x0000000000561a95 in Frecursive_edit () at keyboard.c:821 #20 0x000000000055fba2 in main (argc=1, argv=0x7fffffffe188) at emacs.c:1704 *** bt 2 ** DONE Examine using XComposite rather than GTK off-screen rendering. This would make xembed widgets work much better. This would probably be rathter difficult, but could open up other interesting possibilities for Emacs. There is an early attempt in xwidget.c, but the X call to redirect to offscreen rendering fails for unknown reasons. the attempt was further worked on, and the xlib calls replaced with gdk calls, this works better. In the end I abandoned this aproach. Xwidget-osr is the new aproach. ** TODO make the keyboard event code propagation code work. There is an attempt to provide an api to send keyboard events to an xwidget, but it doesnt currently work very well. *** TODO try gtk event creation instead since that works fine in the webkit osr code. but, oh no, that didn't work for some reason. the event seems to receive the event but then the embedded widgets hangs. *** TODO examine some library to synthesise events xdotool xte xautomation crikey libxdo ** DONE remove the special-case for when the minibuffer is active. I added some code to reduce the annoying problem display artefacts when making the minibuffer the selected window. This made xwidgets in the buffer go grey or black whenever one did m-x to activate the minibuffer. The coded tried to handle the minibuffer as a special case. That simply wasnt a good idea. Special-casing will never work properly. It is much better to spend time finding solutions that work acceptably in the general case. ** DONE disable emacs cursor drawing on top of an active xwidget. This ought to be rather simple and should improve the visuals a lot. ** TODO improve the xwidgets programming interface so its less of hand-waving affair. This shouldnt be too hard, but I have deliberatley not spent any time on it, since getting the visuals right is much harder. Anyway, I sort of think the interface should be somewhat like it is, except symbols is used instead of integers. *** DONE use symbols for xwidget types rather than ints CLOSED: [2011-06-27 Mon 12:52] *** TODO better lisp based structure for xwidgets the lisp interface woud be like this: - make-xwidget returns an xwidget object, similar to a process object. this object is used when creating the display spec(instead of the user defined id now used) the data structure would be something like this: - a "process" like aproach to create the xwidgets. xwidgets are coupled to buffers, somewhat like processes, except a buffer can hold several xwidgets - an xwidget has a plist to hold the model, like a process - an xwidget has an assoc list of xwidget views there are some things that arent clear: - an xwidget doesnt necessarily need to be coupled to a buffer but it seems to be the clearest model. xwidgets would be buffer local - xwidget-views are by necessity coupled to a emacs window so it might be better to store them window locally rather than in an assoc coupled to the xwidget model - for some gtk widgets that resist an mvc approach, like the webkit widgets, special operations are needed, similar to the old phantom widgets aproach. so we need to differentiate live and phantom instances for these troublesome widgets and let lisp manage all the trickery. stuff that needs to work: - do something for all views of a xwidget(resize, value change) - do something for all xw-views in an emacs window(deletion etc) - lookup xw-view for xwidget in emacs window(during redisplay) (- do something for all siblings of a xw-view. not atm) *** DONE xwidget creation interface CLOSED: [2011-07-18 Mon 01:59] xwidgets are a little bit like emacs processes but also a little bit like emacs images. Therefore its not perfectly obvious how to handle creation. Currently I just use hardcoded identifiers. the real scheme needs to be something else. Heres a tentative approach: - xwidget-create returns a xwidget object, like process creation functions. the xwidget will be largely uninitialized until discovered by redisplay. an xw belongs to a buffer - xwidget-insert inserts the xwidget in a buffer. when discovered by redisplay it will be initialized and a xwidget-view allocated - an event will be emitted when initialization is finished when relevant like for sockets the problem with this aproach is that its not really legal to reuse xwidget objects by writing several display specs who reference the same xwidget. It could presumably be done but it would just become weird for no real benefit. the big preblem is that the display spec decides the on-screen size, and its not sane to have xwidget views with different sizes. therefore such display specs would need to be catched and dissallowed. Except it is hard because AFAIK the specs don't have an identity as such. A flag in the structure could be set by lookup so the first display attempt would win. but then you can't rewrite the spec to change the size. hmmm. A third approach would be to just allow the 1st spec refering an xw during a redisplay to take effect, the rest are signaled as errors. this wouldnt be too bad. the other aproach would be to work more like images: - write the display spec with all information needed to create the xwidget. - retrieve the xwidget objet from the spec with an xwidget-at-point function. It can be uninitalized which client code must handle. Unlike assynchronous process creation we dont get back a handle, because there is none yet. - emitted event on initialization, when needed. Many widgets don't need this. for instance, a button sends an event when pressed. but you can't press it unless its on screen, and then its initialized properly. This approach seemed good, but how do I know which instance generates an event if I cant set the id beforehand? so, therefore, the first two aproach is used. *** DONE xwidget creation interface actually CLOSED: [2011-07-18 Mon 01:59] conclusion of above ramblings: - should be similar to make-text-button - don't init from display spec, instead during make-xwidget call *** TODO callbacks would be nice but they need to be handled initially with events for technical reasons. C code can't call Lisp easily. The event handler can call the callback though. ** TODO more documentation There should be user docs, and xwidget contributor docs. The current README is all contributor docs there is now, apart from the code. ** CANCELLED look into more ways of displaying xwidgets, like binding them to a CLOSED: [2011-07-05 Tue 11:34] window rather than a point in a buffer. This was suggested by Chidong. This would be a useful addition to Emacs in itself, and would avoid nearly all display issues. I still think the general case is more interesting, but this special case should also be added. The xwidget would then be bound to replace the view of a particular window, and it would only show in that window. I got the webkit xwidget to work well enough so I dont see the need for this now, except for sockets and I think it can better be dealt with at the lisp level. ** DONE MVC mode for xwidgets CLOSED: [2011-06-27 Mon 12:53] It appears unfruitful to chase using the same display mode for all types of xwidgets. Composition is fun but not robust the way I'm tried to do it. Instead there should be a set of MVC xwidgets. Each on-screen instance of an MVC widget would be a real GTK widget. The instances would communciate state using signals. There are drawbacks. There is no inbuilt support for MVC in GTK, so we have to roll our own, which is tedious if not much work for the few cases. MVC for xembedded application will need support from the applications themselves. Inkscape supports multiple views to the same document, other programs don't. In practice it might not be a big drawback. *** DONE figure out what to do with the multiple frames case. CLOSED: [2011-06-27 Mon 12:52] This should be easier to solve with MVC. Surprisingly, this just worked! *** DONE how to propagate changes in views to other views? CLOSED: [2011-06-27 Mon 12:53] I used gtk signals, the implementation for sliders works well! ** TODO canvas support heres an interesting comparision of gtk canvases http://live.gnome.org/ProjectRidley/CanvasOverview *** goocanvas goocanvas is a gtk canvas implemented using cairo. investigate. pros: - it has a MVC model aproach out of the box which is nice. http://developer.gnome.org/goocanvas/unstable/goocanvas-model-view-canvas.html export CFLAGS="`pkg-config --cflags goocanvas` -DHAVE_GOOCANVAS" export LDFLAGS=`pkg-config --libs goocanvas` ./configure make I made a hello goo world xwidget so seems doable. I wanted to load a SVG which wasnt immediately straightforward, so I tried clutter. but it turns out the exact same strategy could be used with goocanvas. *** clutter maybe clutter can be used as a canvas? pros: - seems to have a lot of traction atm. many examples - potentialy fast and cool vector graphics cons: - no out of the box MVC support, but seems doable. no worse than the other home brew mvc support I have in xwidgets (media-explorer in an application that employes the MVC pattern) http://www.openismus.com/documents/clutter_tutorial/0.9/docs/tutorial/html/sec-stage-widget.html there is also cool stuff like this: http://gitorious.org/webkit-clutter/webkit-clutter which is an webkit actor for clutter! hmmmmm. I want to render svg. aparently: librsvg rsvg_handle_render_cairo(h, cr); ClutterCairoTexture Clutter export CFLAGS="`pkg-config --cflags clutter-gtk-1.0` -DHAVE_CLUTTER" export LDFLAGS=`pkg-config --libs clutter-gtk-1.0` ./configure make compiles but I get: Gtk-ERROR **: GTK+ 2.x symbols detected. Using GTK+ 2.x and GTK+ 3 in the same process is not supported export CFLAGS="`pkg-config --cflags clutter-gtk-0.10` -DHAVE_CLUTTER" export LDFLAGS=`pkg-config --libs clutter-gtk-0.10` ./configure make *** webkit html 5 expose the DOM to lisp or something ** DONE mvc code crashes after a while CLOSED: [2011-07-12 Tue 18:52] seemingly only when compiling with optimizations. I have no idea why. Doesn't seem to happen after some code cleanups. ** DONE xwidget-resize-at CLOSED: [2011-07-19 Tue 14:28] reimplement so display spec is not involved ** DONE display spec validation CLOSED: [2011-07-19 Tue 14:44] it is an error to reuse xwidgets in several buffers or in the same buffer. how do we catch these errors? - showing the same xwidget twice in a buffer is no more wrong than showing in several emacs windows, just conceptually wrong, so ignore this case for now - xwidgets now store a reference to the buffer they were created in, so use that to invalidate xwidget references in oher buffers. but thats not really an error either - xwidgets should now be proper lisp objects so you dont delete them you await their garbage collection. so therefore there can never be invalid disploy specs so turned out this got solved by using proper lisp objects for xwidgets. yay! ** DONE clipping of controllers CLOSED: [2011-07-05 Tue 11:33] Emacs uses a big GTK window and does its own clipping against Emacs windows inside this area. So, in order to layout gtk widgets in emacs windows we must clip thim ourselves. The following method worked well for a long time: - make a gtk widget, say a button, xw - make a clipping area, of type gtkfixed(many types have been tested) - put the clip area in the main emacs gtk window - figure out clip area changes during emacs redisplay the only weirdness was that one has to tell gtk the clip area has a window in order to get clipping. This is weird because all gtkwidgets are windows in a sense and a window is almost by definition also a clipping area. Anyway, in GTK3 the gtk_widget_set_has_window(GTK_WIDGET ( xv->widgetwindow), TRUE); call is ignored. The gtkeventbox which is documented to have its own window doesnt work either. http://www.lanedo.com/~carlos/gtk3-doc/chap-drawing-model.html anyway clipping is rather complicated but seems to finally work okay. *** DONE subclass my own clipping widget CLOSED: [2011-07-04 Mon 16:55] http://www.lanedo.com/~carlos/gtk3-doc/GtkWidget.html#gtk-widget-set-has-window mentions that it has_window can only be called inside a widget impementation. this wasnt really the issue. allocation was the problem *** DONE try scrolled window CLOSED: [2011-07-01 Fri 10:56] clipping does in fact work with gtk_scrolled_window_add_with_viewport (xv->widgetwindow, xv->widget); !! I get unwanted scrollbars in the widget though. gtk_scrolled_window_set_policy ( xv->widgetwindow, GTK_POLICY_NEVER, GTK_POLICY_NEVER); stops clipping from working! *** DONE try viewport CLOSED: [2011-07-01 Fri 10:56] gtkviewport is used in scrolled window so in order to remove scrollbars it should be possible to use viewport directly. however, viewport ignores size requests. or rather the container does. *** DONE debug allocation CLOSED: [2011-07-04 Mon 16:56] the container determines how much size to allocate to child widgets. GtkAllocation galloc; gtk_widget_get_allocation(GTK_WIDGET (xv->widgetwindow), &galloc); printf("allocation %d %d , %d %d\n", galloc.x,galloc.y,galloc.width,galloc.height); after my clipping attemp shows that my size request is ignored! this might be logical, since the container provided by emacs is a gtkfixed. gtkfixed might choose to heed the widgets size desires and allocate the entire widget size. but we want clipping! since i cant reasonably expect to change the emacs main container, i can maybe overide the setallocation method in gwfixed, and adjust allocation to clipping if its an xwidget asking for allocation. **** DONE subclass gtkfixed CLOSED: [2011-07-04 Mon 16:56] possibly i need to subclass gtkfixed and override void gtk_widget_size_allocate (GtkWidget *widget, GtkAllocation *allocation); http://developer.gnome.org/gobject/stable/howto-gobject.html turns out emacs already does this for gtk3 according to jan D: >>For GTK3, Emacs already subclasses GtkFixed, see emacsgtkfixed.[ch]. - widgets may not be underallocated, aparently http://mail.gnome.org/archives/commits-list/2011-April/msg10950.html - how to call base class method/chain up http://developer.gnome.org/gobject/stable/howto-gobject-chainup.html - the allocation modification could happen in the container or the child. it feels more apropiate in the container it is however unexpectedy inconvenient to modify allocation because the needed data is private to the base class. to overcome this: - run base class method 1st. - then, iterate all children, and modify allocation for xwidget children only. x y will then be set. JanD pointed out the GTK3 port already has its own subclass, so I modified that one. *** DONE clip top CLOSED: [2011-07-05 Tue 11:30] there are four controller edges that potentialy need clipping. I begun with right and bottom edges. clipping them is just a matter of setting the right size of the widgetwindow and also ensure it gets the right allocation from the container. clipping top (and left) is not equally straightforward. I'm using a viewport now and scroll it the amount that needs to be clipped. however, the viewport is sensitive to changes in allocation, which makes it harder to use the allocation workarounds. see: - gtk_widget_set_size_request - gtkscrolledwindow I returned to using a simple gtkfixed for the widgetwindow. with allocation hack and set_has_window it works. Idea prefer not to have the allocatien hack and it wasnt needed it gtk3 only gtk2. needs furthi investigation, ** various code cleanups There are many cleanups necessary before any hope of inclusion in Emacs trunk. To begin with, the part of the patch that touches other parts of emacs must be very clean. *** DONE use FRAME_GTK_WIDGET (f) CLOSED: [2011-07-20 Wed 20:02] rather than gwfixed. *** DONE support configure CLOSED: [2011-07-12 Tue 18:48] *** TODO ifdef all xwidget code so you can reliably disable the code at compiletime ** DONE translate clicks CLOSED: [2011-07-03 Sun 22:12] on onscreen webkit peer to offscreen maybe http://developer.gnome.org/gdk/stable/gdk-Windows.html#GdkWindow-from-embedder turned out to be not so hard, captured events, copied them and forwarded them offscreen! ** TODO investigate gdk_window_redirect_to_drawable http://developer.gnome.org/gdk/stable/gdk-Windows.html#gdk-offscreen-window-set-embedder maybe could be used in place of my own copy hacks? to work it must support a chain of redirects, which seems unlikely. the benefit would be that I dont have to spend time optimizing redrawing. ** DONE remove xwidget_views when emacs window is deleted CLOSED: [2011-07-05 Tue 11:29] removing xwidget views when an Emacs window closes is not reliable. - switching buffers in a window seems to hide the corresponding xwidget-views properly, but they might as well be deleted. - patching delete-window-internal could be used to delete the xwidget-views this seems to work ** browser xwidget although embedding a browser is not my primary concern many are interested in this. some suitable browser component needs to be found supporting gtk. *** DONE webkit CLOSED: [2011-07-03 Sun 22:13] there is a webkit gtk port. there is no obvious mvc support. http://live.gnome.org/WebKitGtk http://webkitgtk.org/ it might be possible to keep a set of webxits in artificial synchronisation by recursive deep copy of the DOM from one webkit to another. This will be error prone at best though. Another way might be to just use bitmap copy of the "live"instance to the "phantom" instances. the problem of transfering the live view remains though. export CFLAGS="`pkg-config --cflags webkit-1.0` -DHAVE_WEBKIT -g" export LDFLAGS=`pkg-config --libs webkit-1.0` ./configure make **** off screen rendering export CFLAGS="`pkg-config --cflags webkit-1.0` -DHAVE_WEBKIT_OSR -g" export LDFLAGS=`pkg-config --libs webkit-1.0` ./configure make works a little bit but i get errors like: (emacs:8362): GLib-GObject-WARNING **: invalid cast from `GdkOffscreenWindow' to `GdkDrawableImplX11' set a breakpoint in g_log, backtrace seems to indicate webkitViewportAttributesRecompute is the offender. maybe try gtk3 variants? export CFLAGS="`pkg-config --cflags webkitgtk-3.0 ` -DHAVE_WEBKIT_OSR " export LDFLAGS=`pkg-config --libs webkitgtk-3.0 ` ./configure --with-x-toolkit=gtk3 make crash in gtk_window_get_size instead. great. http://gtkplus-p3.0.sourcearchive.com/documentation/2.91.5-0ubuntu1/testoffscreenwindow_8c-source.html after many atempts, the basic issue remains. for some reason the offscreen widget isnt ok when I want to snapshot it, so i simply get emptiness. the surface is only ok someimes. here is a useful debugging snippets: // debugging redraw: // - the bg colors always change, so theres no error in signal handling // - i get this error now and then: //(emacs:7109): GLib-GObject-WARNING **: invalid cast from `GdkOffscreenWindow' to `GdkDrawableImplX11' // seems to happen in webkit actually. see README if(0){ //redraw debug hack. helped a lot in fact. use the with alpha painter below also cairo_set_source_rgb(cr, osr_dbg_color, 1.0, 0.2); cairo_rectangle(cr, 0,0, xw->width, xw->height); cairo_fill(cr); osr_dbg_color+=0.1; if(osr_dbg_color>1.0) osr_dbg_color=0.0; } you need to terminate drawing like this: //cairo_set_source_surface (cr, src_pixmap, 0,0); //cairo_set_operator (cr, CAIRO_OPERATOR_OVER); //cairo_paint_with_alpha (cr, 1.0); //cairo_paint(cr); the snippets change background color on oach redraw. **** on-screen rendering to separate window an alternative might be to open a separate window and snapshot it. the idea is that whatever oddness webkit does so that offscreen rendering doesnt work, doesnt happen on-screen. the window could be opened somewhere not in the way. *** CANCELLED firefox CLOSED: [2011-07-03 Sun 22:13] http://www-archive.mozilla.org/unix/gtk-embedding.html seems to be severly bitrotted heres a newer aproach http://hg.mozilla.org/incubator/embedding/file/29ac0fe51754/gtk/tests/test.cpp while webkit clearly has the best traction as an embeddee, the offscreen rendering issues makes it interesting to see what ff brings to the table. turned out webkit has as good offscreen support as anyone, see I went with that in the end. *** DONE text field support CLOSED: [2011-07-20 Wed 20:05] Emacs captures all keyboard events so text field support isn't super straightforward. **** propagate keyboard events I have some old hacks for this and they are not good. **** use the DOM model expose document.activeElement to lisp. This is potentially more interesting than just forwarding keyboard events. webkit_web_view_get_dom_document () this is hard it seems. an idea might be to hack elisp support for swig to machine generate the bindings. **** DONE inject javascript CLOSED: [2011-07-03 Sun 22:50] webkit_web_view_execute_script () this works now: (xwidget-webkit-execute-script 5 "document.activeElement.value='test'") so it should be possible to do some interesting stuff. execute-script does however not return anything at the interface level so satisfaction is not total: http://markmail.org/message/4yowmdgras73z3x5 maybe https://launchpad.net/gnome-seed or this funny hack: im trying to understanh how to interact via javascript to an embedded webkit gtk instance [23:38] i use webkit_web_view_execute_script() which is nice but doesnt return a value, by design aparently [23:39] any hints? jave: afaik, webkit still doesn't have full gobject bindings [23:48] jave: you can hack it up by making the JS modify the title, and read the title from gtk-side lucian: that was a pretty cool idea! *** webkit_web_view_load_string () I would like preview of html in a buffer rather than from uri. *** DONE simple xwidget-webkit wrapper CLOSED: [2011-07-22 Fri 11:01] so that it could be used for actual browsing :) I dont want to reinvent too many wheels so i'd like to use existing emacs facilities here possible. use bindings similar to w3m(or info) - m-x xwidget-webkit starts a session - 'g' goes to a url - use bookmark-jump i suppose. I mostly use org for bookmarks myself - browse-url support so webkit can be the default browser - some way of getting around the quirky keyboard interaction since xwidgets dont receive keyboard events because I hawe no idea how to do that in a sane way ... and one can of course go on bikeshedding forever. lets keep it simple and extensible, and compatible with other Emacs packages. the really cool ideas would need Emacs DOM integration, which is not easy. ** webkit related *** TODO webkit support webkit signals **** DONE particularily document-load-finished CLOSED: [2011-08-01 Mon 22:34] http://webkitgtk.org/reference/webkitgtk-webkitwebview.html#WebKitWebView-document-load-finished because one might need tell set a title and sizes and things when it loads. **** TODO event bug Debugger entered--Lisp error: (error "Two bases given in one event") hapens sometimes with xwidget events. appears to be when the originating xwidget is offscreen so that the event doesn't get caught by the correct emacs event map. maybe I need to set the originating window in the event structure. event.frame_or_window = Qnil; //frame; //how to get the frame here? //TODO i store it in the xwidget now since its an offscreen xwidget the buffer local keymap isnt the right place for the handler. some global map should be used. onscreen widgets don't have the same issue. anyway, seems it'll turn out like this: - xwidget-osr stores a callback and user data - the event is an implementation detail only and get caught in the topmost event map - the event map calls the callback in the xw with the right args. we need the event handler at some level because we can't call lisp asynchronously. *** TODO console messages http://webkitgtk.org/reference/webkitgtk-webkitwebview.html#WebKitWebView-console-message http://getfirebug.com/wiki/index.php/Console_API#console.count.28.5Btitle.5D.29 because maybe we can make a simple JS REPL that way. (xwidget-webkit-execute-script ( xwidget-webkit-last-session) "console.log('hello')") prints hello to stdout but theres no way to catch stdout from webkit I think other than receiving the signal. *** TODO webkit flashkiller by default while its possible to support plugins in the webkit xwidget, flash has issues on 64 bit, and slows down emacs to a halt with off screen rendering, and of course is not free software. its in the way for real world usage even if its interesting to watch flash animations inside emacs. which should be achieved with Gnash or other free software instead. http://stackoverflow.com/questions/4885513/prevent-flash-in-cocoa-webview simply use this api: http://webkitgtk.org/reference/WebKitWebPluginDatabase.html theres an implementation now but it's not robust enough webkit often crashes taking emacs with it. *** TODO webkit downloads when clicking a download link in Webkit Emacs should take over and handle it from there. Probably need signals. There are Emacs libraries to download things, with wget etc. "download-requested" *** TODO webkit alt-text not handled XKCD use image-title to display a cartoon comment. These mysteriously don't work ATM. Other mouseovers work though. Maybe webkit tries to open a new window or something, which wont work. *** TODO webkit isearch in webkit buffers have a look at how docview solves it webkit_web_view_search_text () *** TODO webkit relative references doesn't work probably because we handle scrolling in a non-standard way. It does work sort of when theres a html frameset and webkit scrols by itself. *** TODO webkit width adjustment handling issue since there are so many levels of clipping and whatnot in xwidgets sizing issues are difficult. - an xwidget is told how large it can be by emacs. thats the end of it. if the xwidget thinks otherwise it will be clipped. - but emacs can ask the xwudget how large it wants to be. it can then resize the reserved area and inform the xwidget thusly. That should have been enough. but webkit never reports less than what it already has. So currently a webkit view will only growth and not adjust to smaller sizes. This is not a big problem in practice but is still annoying. to see the problem surface to http://www.slashdot.org - xwidget-webkit-adjust-size - xwidget-webkit-adjust-size-to-content and then compare by resizing in Epiphany, which is also webkit based. *** TODO xwidget webkit allow loading from string from emacs *** DONE xwidget-webkit-last-session CLOSED: [2011-08-01 Mon 22:38] was rather hurried. end result is that the lisp layer only really allows for one webkit session. *** TODO C-X b in other buffer from webkit bafflingly resets the webkit view to the top. Maybe the window reconfiguration hook code? further mystification is added because it only seems to happen with ido mode enabled. in comparison with image-mode which does the right thing, I discovered that image-mode has special code to handle scrolling. the browser mode and image mode has some similarities. ** TODO xwidget image display spec compatibility some history: the first version of the xwidget display spec was the same as an image spec. This turned out not to be fantastic because an xwidget is both like a process and like an image. it has a separate existence from display. So now the xwidget display spec is just a pointer to a xwidget. But then some useful functionality in Emacs can't be reused for xwidget, in particular image-mode. Maybe a new image type could be added that was a wraper on an xwidget. Then image mode could be reused for webkit mode. ** socket related *** TODO some flickering during redisplay of sockets with gtk3 an size allocation workaround is used. this seems maybe to result in flickering sizewize y-axis with the xwidget socket type. The webkit xwidget doesn't seem similarily afflicted. the size allocation workaround works by 1st running the ordinary allocation then modifying the results. its done this way to minimise the copy paste index from the base class. it might be that the original allocation has a brief time window to show itself. tried to modify the allocation hack so it doesn't call allocate twice. this doesn't seem to help flicker at all aparently so the hypothesis falls. Maybe then a socket simply doesn't lke being clipped by gtkfixed. *** TODO xwidget view reaping too agressive hide an emacs window for a while and return to it. the xwidget might get reaped and a new socket thus created. *** DONE try out OSR for sockets CLOSED: [2011-07-25 Mon 21:30] didn't work too well in the inkscape case. it might be that some other bitmap copy method works better though. ** TODO synchronise emacs background with xwidget color fine-tuning to reduce flicker. ** DONE xwidgets doesn't work during bootstrap all of a sudden CLOSED: [2011-08-01 Mon 22:33] might be some annoying local issues with my install because it is not reliably reproducible. (went away during merges) ** TODO low impact xwidget based image viewer for instance to render SVG using webkit, or some other canvas. that way it would be possible to merge to trunk in stages. so, webkit could be used to display the SVG. the display spec for images would be used. multiple webkits would be used rather than offscreen rendering, so it would be GTK2 compatible. ** TODO xwidget movement doesn't work all of a sudden this used to work great. now it doesn't. suspects: - XCopyArea - x_shift_glyphs_for_insert - x_scroll_run. this is run by the try_window* functions, and inhibiting them dösnt help. but also callid in scrolling_window. - try_window_reusing_current_matrix - I used to enable GLYPH_DEBUG which I currently don't. it disables many optimisations. this was fixed. - lookup_xwidget then produce_xwidget_glyph gets called always but not x_draw_xwidget_glyph_string probably because of scroll optimization. movement detection could possibly be moved to produce_xwidget_glyph(not) no longer helps: (setq inhibit-try-window-id t) (setq inhibit-try-window-reusing t) workaround: (run-with-timer 1 1 'redraw-display) seems to work: inhibiting scrolling_window(). and this seem to be enaugh to restore old behaviour, GLYPH_DEBUG doesn't seem needed. ** DONE GLYPH_DEBUG doesn't work CLOSED: [2011-08-08 Mon 17:30] was stupid accidental line removal that was hard to spot * old notes from x_draw_xwidget_glyph_string BUG it seems this method for some reason is called with bad s->x and s->y sometimes. When this happens the xwidget doesnt move on screen as it should. This mightbe because of x_scroll_run. Emacs decides to scroll the screen by blitting sometimes. then emacs doesnt try to actualy call the paint routines, which means this here code will never run so the xwidget wont know it has been moved. Solved temporarily by never optimizing in try_window_reusing_current_matrix(). BUG the phantoming code doesnt work very well when the live xwidget is off screen. you will get weirdo display artefacts. Composition ought to solve this, since that means the live window is always available in an off-screen buffer. My current attempt at composition doesnt work properly however. //allocation debugging. the correct values cant be expected to show upp immediately, but eventually they should get to be ok // this is because we dont know when the container gets around to doing layout //GtkAllocation galloc; //gtk_widget_get_allocation(GTK_WIDGET (xv->widgetwindow), &galloc); //printf("allocation %d %d , %d %d\n", galloc.x,galloc.y,galloc.width,galloc.height); *** old notes about the old live/phantom scheme //TODO: // 1) always draw live xwidget in slected window // (2) if there were no live instances of the xwidget in selected window, also draw it live) // 3) if there was a live xwidget previously, now phantom it. else { //ok, we are painting the xwidgets in non-selected window, so draw a phantom //printf("draw phantom xwidget at:%d %d\n",x,y); //xwidget_composite_draw_phantom (xw, x, y, clipx, clipy); //TODO MVC there will be very few cases of phantoming } atm this works as follows: only check if xwidgets are displayed in the "selected window". if not, hide them or phantom them. this means valid cases like xwidgets being displayed only once in non-selected windows, does not work well. they should also be visible in that case not phantomed.