Web Site and Code Management Tools
|
|
Web Site and Code Management Tools |
|
|
(This is hardly more than a draft and shouldn't be up here for the world to see, but I'm excited about it, so I'll put it up anyway, -- dpw)
This is a brief analysis and a somewhat longer design for a tool which will help organize text, software, and web page components such as HTML code, scipts, downable source and binary archives, images, audio, video, databases, and anything else that could be put together as a modern website. It owes something to the familar make utility used for software, and something to the website content management systems used to create and maintain large websites.
A prototype is already under development and should be available soon since it makes use of a lot of existing code. This code and whatever is added to it will be made available as open source software on the web, and it seems like a very good idea to get this project written up and installed on SourceForge.net even before the prototype is finished.
It is worth mentioning that the motive for this project was the rapidly growing website for SocialTechnology.ca that is being put together by Doug Wilson. He also has a personal website that is just as badly in need of organization, and large amounts of software source code and ordinary plain text, written over a couple of decades as part of a vast project that is only now becoming visible on the web.
a website creation and maintenance system that will be object oriented with a class hierarchy. Pages similar to one another can all be created in a uniform way by including material from a parent or superclass. As is often the case, the key design idea arrived early on, before the requirements analysis was properly done. As design went on some requirements that should be considered evolved. It is probably better to address those here, first, before going into the design details.
This program or system has been suddenly envisioned as a way to simplify the creation and maintenance of web pages, but it should be observed that these web pages include a lot of material that was originally written for a book. It should still be possible to have it as a book, and this piece of software should encourage keeping that option open.
Several scenarios are envisioned for the working system. The most basic one is a system for putting a lot of text up on the web with very little HTML work. The text will simply be collected from various places then dispersed into new locations appropriate for this piece of software. The possibility of keeping it where it is and having it pointed at instead of moved should be kept open.
A bit of design intrudes here. The eventual organization of material is intended to be a tree of nested directories. It should be possible to have this simulated by pointers or links to existing material in existing directories, which, however can be moved into the newer structure when necessary.
This system makes a lot of directories, since its object-oriented features depend on the existence of a directory hierarchy with one directory for each webpage to be created and maintained. In addition, node of the directory tree higher than the leaves of the tree will each require a separate directory. A website containing 100 web pages may therefore require a directory tree containing somewhere from 110 to 150 nested directories.
However, these directories will contain few files and not take up much space on the disk --- by eliminating the need for multiple copies of common components of the pages the total amount of diskspace used may well be considerably less than if the pages were stored individually in their final form.
To build a page, the program goes into the directory dedicated to that page and looks for a makefile. If there is a make file, it is used and as a result the contents of that directory may grow by one or more file. Most directories will not contain make files, since this system is not based on such an approach and doesn't need it. But for extra flexibility it seems a good idea to allow for a file creation step just before the assembly of components into a web page.
If there is a file creation step, in many case that will simply be the execution of a text-to-html program to make the body of an HTML page from a text file. If the directory contain no HTML body file, nor makefile to create one, but does contain an ordinary text file with some appropriate extension, that file will be transformed into HTML by applying whatever text-to-html program is available or is specified somewhere.
After the file creation step has been performed or bypassed, depending on the presence of a makefile, the program then goes into inheritance mode where missing files are replaced with a reference (not a copy) to the corresponding file in the parent directory. Since this directory is probably one intended to contain some real content, the central file that is most likely present is the file containing the actual body of the (normally HTML) document, this directory probably will contain the body of an HTML document, but it may not contain anything else. Many leaves of the directory tree will inherit the other (header, footer, etc.) from the parent node.
The first step in building a page from its directory is to find the components file which lists the components of the page and the order in which to assemble them. A components file might contain simply this:
header
body
footer
in which case the program will concatenate a header file, a body file, and a footer file to produce the page. If any of the files specified in the components file does not exist, the program will seek a file of that name in the parent directory. If it is not found in the parent directory, it will seek it higher up the directory tree. There should be at least one file of each name in the root directory for the website, but if not the program can create one -- but only after warning and prompting the user.
The components file itself might be missing from a webpage's directory, but again that can be provided by searching up the directory tree.
The system comes with an initialization program that creates a directory tree from an indented plain text file. Typically this file is created by the user by hand using a text editor, and it might look something like this:
root navigate index sitemap content forms text overview links
Of the 8 directories listed here, three of them, root, navigate, and content probably will not correspond to web pages, but are simply classes. Under navigate, index and sitemap are probably directories for actual pages, but under content the three directories forms, text, and content are probably classes, with only overview corresponding to an actual web page.
The system could generate the pages in any order, but to include the possibility of having a simple linear order from first to last, like the chapters of a book, an ordering system will be used. Since this program is much more general than it need be, it could actually be used for assembling the chapters of a book. Therefore it should use a depth-first left-to-right traversal of the directory tree. The possibility of starting each node of the tree by a breadth first collection of summaries to make a hierarchical table of contents should be considered as well.
It should be possible to run the program on the same directory with different options selected and generate a website in one case and a book in another.
and still further thoughts ...
The prototype will probably just put things in an actual directory hierarchy, but really the actual directory structure, as known to the operating system, should not be a necessary part of this, and should not necessarily be visible to the higher level parts of the software.
In particular, it is well known that the hierarchy is not a very natural form of organization and that the same material could just as well be organized in many different hierarchies. That should be kept as an open possibility. As a design note, perhaps the descent of a parent into a child node should be accompanied by some kind of key which tells the program or component working at the child level which particular hierarchy this particular assembling operation is workinb from.
Linear ordering as TSP should be noted and provided for, as well as CASA like numerical analysis and combinatorial optimization of many kinds. Graphics for visualization should be added.
But this extra functionality should be added, not built into the very first proto-proto-prototype. So write a bit of crud-ugly crude code and try it out.
... and that's it -- more work on this to come soon
New: Social Technology through Diagrams
New: Social Techs novel online
The main Social Technology page.
Find Compatibles, the key page, with the real solution to all other problems explained
Technological Fantasies , a page about future technology
Social Tech a page about Social Technology, technology for social purposes. I think I was the first person to use this phrase on the Internet, quite a long time ago.
Roughly corresponding to these web pages are the following blogs:
Social Technology the main blog, hosted on this site, with posts imported from the following blogger.com blogs, which still exist and are useable.
Find Compatibles devoted to matching people with friends, lovers, jobs, places to live and so on, but doing so in ways that will actually work, using good math, good algorithms, good analysis.
Technological Fantasies devoted to future stuff, new ideas, things that might be invented or might happen, such as what is listed above and below.
Sex-Politics-Religion is a blog about these important topics, which I have been told should never be mentioned in polite conversation. Alright that advice does seem a bit dated, but many people are still told not to bring up these subjects around the dinner table.
I believe I was the first person on the Internet to use the phrase Social Technology -- years before the Web existed.
Those were the good old days, when the number of people using the net exceeed the amount of content on it, so that it was easy to start a discussion about such an upopular topic. Now things are different. There are so many web pages that the chances of anyone finding this page are low, even with good search engines like Google. Oh, well.
By Social Technology I mean the technology for organizing and maintaining human society. The example I had most firmly in mind is the subject of Find Compatibles, what I consider to be the key page, the one with the real solution to all other problems explained.
As I explained on my early mailing lists and later webpages, I find that social technology has hardly improved at all over the years. We still use representative democracy, exactly the same as it was used in the 18th century. By contrast, horse and buggy transporation has been replaced by automobiles and airplanes, enormous changes.
In the picture below you will see some 18th century technology, such as the ox-plow in the middle of the picture. How things have changed since then in agricultural technology. But we still use chance encounters, engagements and marriages to organize our home life and the raising of children.
I claim that great advances in social technology are not only possible but inevitable. I have written three novels about this, one preposterously long, 5000 pages, another merely very very long, 1500 pages. The third is short enough at 340 pages to be published some day. Maybe. The topic is still not interesting to most people. I will excerpt small parts of these novels on the web sometime, maybe even post the raw text for the larger two.
This site includes many pages dating from 1997 to 2008 which are quite out of date. They are included here partly to show the development of these ideas and partly to cover things the newer pages do not. There will be broken links where these pages referenced external sites. I've tried to fix up or maiintain all internal links, but some will probably have been missed. One may wish to look at an earlier version of this page, rather longer, and at an overview of most parts of what can be called a bigger project.
Type in this address to e-mail me. The image is interesting. See Status of Social Technology
Copyright © 2007, 2008, 2009, Douglas Pardoe Wilson
I have used a series of e-mail address over the years, each of which eventually became out of date because of a change of Internet services or became almost useless because of spam. Eventually I stuck with a Yahoo address, but my inbox still fills up with spam and their spam filter still removes messages I wanted to see. So I have switched to a new e-mail service. Web spiders should not be able to find it, since it is hidden in a jpeg picture. I have also made it difficult to reach me. The picture is not a clickable link. To send me e-mail you must want to do so badly enough to type this address in. That is a nuisance, for which I do apologize, but I just don't want a lot of mail from people who do not care about what I have to say.