| Joel |
A discussion on the AGI Studio forum reminded me of the AGI inter-process communication standard that some of us started talking about some time ago. I've gone ahead and come up with the beginnings of a draft of a possible standard. If you're interested in helping to develop this, then look over it and tell me where you think it needs improvement. You can also offer suggestions for the standards I haven't even touched on yet. In the current document, I've basically defined a communication method that would allow a program like AGI Studio to communicate with a program like the Base Logic Generator. If the method I've defined is expanded to a full set of standards, there's a lot more that needs to be defined. Anyway, have a look at what I've got. (by the way, even if you think it's crap, say that -- I don't care if this particular proposal is used...I just want to get things moving on this).
http://weremoose.tripod.com/ComProposal.html
|
| sonneveld |
Hey good work. nice way to get the ball rolling. :)
One suggestion.. instead of flags and version numbers to determine type, why not just use an integer or string? there's more combinations that way.
ie
0=pic edit 1=view edit 5=compiler
or with a string, you could copy the IFF spec and embed the version in the string. ie VIEWEDIT, VIEWEDI1, VIEWEDI2, ....
Also, I'm not intimately familiar with tcp/ip but doesn't tcp/ip look after splitting messages up and lengths for you? Because of the structure, it guarantees that the data's been sent, and if not it gives you an error.
there should also be a Focus command or something similar so that if somebody goes to agistudio, and it's still waiting for a program to finish, it can call this focus command and the called program will pop up instead of agistudio.
are we going to look into having multiple editors open at once?
- Nick
|
| Joel |
Sorry, I should have explained some of my reasoning a little better. The reason I used flags to identify the application type is that I was thinking that some applications might fall under more than one application class. If integers or strings are used, then it makes all the programs have to be aware of a whole range of values. For example, let's say that the number 2 indicates a view editor only, but the number 17 indicates a utility capable of editing views and editing picture resources. In that case, AGI Studio or whatever would have to look at the app. type integer like this:
if (apptype == 2 || apptype == 17, etc.)
{
caneditviews = true;
}
The version number isn't really used to determine the type of application. But if the first three bytes or so of each packet are the same, then the IPC standard can be changed later on down the line (for example, the header formats, except for those first three bytes, might be completely changed) without breaking existing apps. The existing apps would simply know, "hey, i can only handle version 1 packets" and be able to respond appropriately. Which I guess suggests that there ought to be a "version number too high" type of message, or maybe a general error message with maybe a 32-bit error code field.
on TCP, lengths, and splitting messages: yes, TCP does guarantee delivery of a packet or an error (in this case, since it's on the same machine, there's almost no chance of delivery failure unless one of the apps. screws up). However, TCP is a bytestream protocol with no record boundaries, so the length field is very helpful in determining how much data should be received into the current record. TCP might fragment messages over a network for some reason or another during routing, but I don't believe there's a theoretical limit on how large a packet can be. The only real purpose to splitting up the packets is so the length field can be valid. The packets could be made much larger if the length field counted, for example, how many 32-bit words there were in the packet. In that case, the packet could be 65535 * 4 = 262140 bytes each, but I don't really see where it would decrease the complexity any, so I didn't do things that way.
That focus message is a good idea. It would mean extra complexity in the utilities, though, because they'd have to be monitoring for focus messages and such while doing whatever other work they do. It would mean multi-threading almost for sure. I think Windows Sockets let you asynchronously monitor your app's TCP receive buffer, but I don't think other OS's do that without multi-threading (as the simplest route).
I think it's reasonable to allow multiple utilities to be opened at once by AGI Studio (that's one reason why there's a session id field). I think that would really be up to the designer of the program that's using the utility, though.
One difficulty I thought about is when the utility app sends its data back to AGI Studio. I guess it depends on what sort of editing AGI Studio expects the utility to do. Something like the BLG should create source code, send it back to AGI Studio, and then close. But if somebody developed their own AGI syntax-highlighting editor, then that would be a source code generator application that needs to be able to be left open, possibly even to have access to a compiler through the IDE. It wouldn't want to close just for sending data back to AGI Studio. What if the user has been writing for a long time and just feels it's necessary to save? They wouldn't want the logic editor closing on them in that case. But AGI Studio does need to accept and save the data.
Anyway, I'll stop for now before this turns into an essay.
|
| Joel |
one other thing, that focus message would be applicable only if external utilities are treated as a sort of modal dialog box. If we want users to be able to interact with AGI Studio and the utilities at the same time, then automatically shifting focus would get in the way of that kind of activity. Once again, for some apps it makes sense, for others it doesn't. I'll use the same example here: the BLG makes sense to use modally, but an external source code editor wouldn't make sense to use modally. So, with something like the BLG the focus message would probably work well. With the external source editor it wouldn't.
Maybe there should be a field in one of the messages where either the utility or the IDE requests that the utility be run modally, indicating that it needs to repond to focus messages?
|
| sonneveld |
I've been thinking about this a fair bit and why do we need all this trickery to send files to agi studio?
The problem with agi studio is that it always works from the volume files.. you can't have any other format other than the standard agi formats. So we're basically sending files over the network so agistudio can extract and import files back into the volume files for you.
we have to rewrite agi studio for the support anyway, so why not change the way it edits files? every single resource in a game should be a separate file, they shouldn't be put into an agi volume resource until you want to actually play the game.
if you want your view editor to edit a view file, just open up a new instance of it and point it to the file you've got in your game directory. if you want to make a new file, agi studio should make an empty file, add it to the "project file" and then point your editor to the file.
the only thing that really needs communication between apps is debugging I think.. everything else doesn't have much interaction between programs.
- Nick
|
| Joel |
I think Nailhead's input would be valuable on this issue. If changing the way AGI Studio handles files would be relatively simple, then it would probably be easier to do things like that. One thing about the communication over sockets, however, is that the programs can notify each other about stuff, which seems to me to provide greater potential for a more integrated feel with certain programs (such as the BLG). Also, the IPC communication could be used by any AGI program, not just AGI Studio. For example, a picture editor program could implement the IPC communication and use that to communicate with the AGI Repeat program I wrote (using its functionality as a feature the author wouldn't have to write). Using the socket interface, the two programs could synchronize their content to make sure each has the most up-to-date version of a file (as it is now, there's a high potential for losing changes if the two programs are used at the same time). There are other method of accomplishing the content synchronization (many programs monitor their open files to see if they've been modified), but the point is that things of that nature would be possible with the socket interface.
|
| sonneveld |
If you want to edit more than two files, you'll definately have problems with syncronisation. What if both programs have added something to their respective pics before they notify each other? which one do you keep? It would be better to have separate files and lock each one when you open and edit it. You can't really lock a volume file if you want to have multiple programs edit them.
I think if you want to have a picture repeater and a picture editor, it would be best to combine them into one program. The only thing I can think of for communication is if you want to use a logic template generator and you want to find out which program has a file open and if they can close it.
so you'd have a discover command to see which program has a file open.. (if you can't do this with standard windows) and a close command to kill the program which has it open (if it's ok with the user)
The way I was thinking, you could simplify agi studio a LOT and just make it a small project management program and have the view/logic/words.tok/sound whatever as separate programs.
- Nick
|
| Joel |
there are probably ways to implement mutual exclusion with a communication interface, although it would increase the complexity even more -- in any case, it was just an example of a potential use of the communication that isn't possible (at least not without at least as much difficulty) without it. Another would be integrating the BLG so that as soon as it generates a file the file appears in AGI Studio's logic editor. Also, I was thinking along the lines of interchangeability, where someone could write an entirely new IDE (maybe even just a project manager) and the new IDE would be able to integrate utility apps without any need for changes in the utilities. of course, if nobody really sees a need for any of this, then there's not much point in doing it.
I don't particularly care for the idea of having AGI Studio as a project management program with all the other stuff separate, but only because I prefer the integrated development environment interface. I find interfaces like the Windows Linux AGI Studio to be cumbersome.
|
| sonneveld |
well, even with a base logic generator, you could have agi studio produce an empty file, add it to the project and then point ablg to the empty file to overwrite it.
And also, if we have utils like an external pic editor and a blg, they'll all be external tools anyway, so why not make some of the other ones external as well? That way you can easily replace the project manager, or a pic editor because it's all modularised.
I just don't want over engineer everything. It's like the difference between firewire and usb (usb: components can only connect to pc.. firewire: components can connect to components)
And if you just have files and point external tools to files, you'll be able to add any old utility to your ide. just make sure the command line parameters are the same.
- Nick
|
| Joel |
That method seems like it would be adequate for most purposes, as long as Nailhead thinks that change is easier.
As for the editors being separate from the IDE: I don't mind apps like the BLG being separate from AGI Studio because they are quick-use utilities that I use and am done with. Also, I will typically only open one instance of a utility at a time. With editors, however, I frequently open multiple editors at once. PicEdit is already a bit of a nuisance (though that's partially because it's a DOS program, which makes the computer have to switch video modes when I task switch to it). If I were to open several logic or view files at once, I wouldn't want to have several additional windows in the task-switching chain.
The option to use external editors would be fine with me, but I see no reason to eliminate the internal editors and make them into external ones or anything like that. Maybe "no reason" is not quite accurate...yes, it would make modification of the default editors easier, but user interface is an overriding concern, in my opinion.
|
| sonneveld |
Well perhaps keep the current internal editors but allow you to replace them with an external one if you wish.
There's at least one text editor (scite) that lets you open multiple text files in one window, with a tabbed interface.
and how many pictures do you need to edit at a time? :)
- Nick
|
| Joel |
with PicEdit, I actually just meant switching to it at all is kind of annoying -- but like i said that's mostly because of the video mode switches. Admittedly, I don't often want two pictures open at once :), but there have been rare occasions when I've wanted to look at one while drawing another.
I probably wouldn't mind an external editor that allowed me to open multiple resources of the same type in one workspace. It might not be ideal, but the only interface I have a real problem with is the kind where each document in the same project has its own window where I have to move past it to task switch. That's the way Windows Linux AGI Studio does things...and I think the very first versions of SCI Studio did it (back when it was called SCI Graphic Studio). Microsoft Word 2000 does things like that, too (except without projects), and I can't stand it.
|
| sonneveld |
ok, fair enough.
I wish we had more input on this.. seems like we're just designing this for ourselves. :)
- Nick
|
| Nailhead |
Alrighty, I just discovered this thread. Great job on the protocol! I think this is excatly what we need to tie in external AGI utilities. In fact, Nick's idea of using words.tok/sound/picedit as modules for agistudio is a great idea, and using Joel's communication idea seems perfect.
Updating agistudio is a really hard task since I'm constantly trying to make since of Peter's code. What I'd really like to do is to start over. I know, I know, why reinvent the wheel right? It would be more of a cut and paste job really. Agistudio would be a managment system which will include a tabbed text editor interface and maybe a code inspector (for variables and flags). Agistudio would then call an external compiler which Nick is working on, but until it's done I could cut and paste the compiler code into it's own executable. This could be done with each major feature of the current Agistudio. These components will inevitably be replaced by other people (using whichever language they prefer). This will dramatically increase the development of Agistudio, giving everyone a chance to add on to it.
Should we start a poll for this idea and see if this is what everyone wants?
|
| Kon-Tiki |
Hmmm, sounds like the thing we need for 'Cinderella' (only ours should be in SCI <dagnabbit>.) It's a great
idea.
btw: The reason why I ignored this thread before, was because I didn't know what this was all about.
-Kon-Tiki-
|
| sonneveld |
I think we'll have to figure out the options before we have a poll.. and carefully explain each option.
The compiler is coming along.. I'll let you know how it's progressing a bit later I guess. I've got most of it planned out now. You might have to adapt agi studio to accept multiple errors.
I know Joel suggested this in an earlier thread but we'll have to define a standard command line parameter system as well.
like
-e <file> edit file
-n <file> create new file (for template editor)
agi studio could pass along known logic source via -e. or create a temp file like agitemp.logic via -n and ask which logic to save it as afterwards..
the return code could also be used..
0 - success
1 - could not load file
2 - could not create file
3 - canceled
stuff like that
- Nick
|