Although this approach probably won't work for your current 'Sewing Together' project, I wanted to alert you to something that I think the bot can do rather easily, and I would like to try it at some point.
Let's say you have a pattern that you would like to change - for example, someone's lineage was imported incorrectly, and we need to modify it for each of that person's descendants. Let's say John Whitney had been imported and his lineage on all of his descendant's pages incorrectly listed his father as Moses instead of Isaiah.
We could say that we wanted to replace each instance of this:
[[Family:Whitney, John (1698-1765)|John]]<sup>4</sup>, [[Family:Whitney, Isaiah (1671-1712)|Isaiah]]<sup>3</sup>, [[Family:Whitney, Thomas (1627-1719)|Thomas]]<sup>2</sup>,
[[Family:Whitney, John (1698-1765)|John]]<sup>4</sup>, [[Family:Whitney, Moses (1655-a1729)|Moses]]<sup>3</sup>, [[Family:Whitney, Richard (1624-a1691)|Richard]]<sup>2</sup>,
Running that conversion across all Family pages should catch and correct all of his descendant's pages. The next time you identify something like this, please let me know. I'd like to try it to see how well it works. This could be a big time-saver.
Pierce Project Page Collisions
It looks as though at some point we collided with each other on the Pierce Project page edits and the Misc. Section was deleted. With heavy multi-user edits, the following pointers will help to prevent this:
- If you edit a section of the page and the incorrect section is displayed in the edit window, back up, refresh the page, and try again.
- If you attempt to save and a message is displayed indicating that there has been an edit conflict, be very very careful and read the message that is displayed. If you save at that point, one of the edits will be overwritten.
Thanks! It looks like we're making great progress!
The comments at the top of the recently imported pages are an artifact of the import process. Although they do not show up on the displayed page, they are unnecessary and might confuse people who edit these pages in the future. As I edit pages, I have been trying to remember to remove these comments. Feel free to do so as well.
- <!-- '''Family:Whitney, James (1816-1889)''' -->
It looks as though quite a few of our lineage break issues are a result of the alpha ID numbers. It's too bad I didn't think of this before we imported - could have saved us a lot of work!
I can only imagine the painstaking process that Pierce had to go through to put his book together - gathering the data, putting it together manually, creating a numbering system as additional information arrived by mail, causing him to have to decide whether to start over or to squeeze in the new information using these alternate numbering schemes. Although an annoyance, I suppose we should be happy that we can so easily change things on-the-fly.
I am beginning to think about how I will handle the location categories using a bot. Here's what I am thinking:
- Load an array of locations (based upon your geographic index)
- Loop through all Family pages
- Load the current page
- Locate all existing catagories on the page, store them in a list, and remove them from the page contents.
- PERHAPS: Loop through all locations in the master array to see if any others exist which weren't caught in the import
- PERHAPS: Somehow catch locations that have two words, such as 'Lake Placid' and fix them in the list
- Loop through all categories in the list
- Look up location in the array of locations
- Remove existing category from the list and add in the new one (which includes the county)
- Add the county category
- Sort all categories in the list
- Add list of new categories to the bottom of the page
- Save the page
While doing this, I could also remove the comment lines. I'd rather not make a full pass of edits just for the comments - each edit makes the database larger, so combining such edits will keep our database size down.
Census Records on Family Group pages
I am starting to think about how we could facilitate the identification / linking of census records to the family pages. We could just include references to the various enumerations within the text of the family, and that would be the traditional route. However, it seems as though that would be difficult to visually see who had been identified, what was left to be identified, etc. Take a look at User:Tdoyle/Sandbox2 to see one suggestion of how we might handle this. Adding this section with appropriate census years to be identified could be handled via the bot and then populated manually.
I have just completed the following mass changes using the bot:
- "Chas. " -> "Charles "
- "Fred'k " -> "Frederick "
- "Fred'K " -> "Frederick "
- "Geo. " -> "George "
- "Jas. " -> "James "
- "Jos. " -> "Joseph "
- "Thos. " -> "Thomas "
- "Wm. " -> "William "
- "WM. " -> "William "
If you find any other such abbreviations, please let me know and I will correct them as well. If you find any issues with the above, please let me know.
Auto Lineage Corrections
Go to recent changes and select 'Show bot edits'. See the recent edits there correcting the links to Joshua? This is so much faster than manually fixing such issues. Feel free to send me any such change requests. Also, did you notice Mike Poston created a page for Jabez Whitney? Woo hooo!
I have now generated a proposed list of places to be converted on the family pages and have placed it on Pierce Conversion Project - Place List. Please review this list and let me know if you see any issues.
Note that I do intend to add category links to the "county, state" as well as the "city, county, state".
If there are any changes which would be solved more easily by making a programmatical change and re-running the script I wrote to create the page, then any changes you've made to that page would be lost. At some point, when we've ironed out any major issues, then editing that page would be appropriate. Feel free to add comments at the top of that page through for any items found.
Whitneys of Clifford
While putting together a page for the Whitneys of Clifford, I think I have stumbled across a piece of information that lends substantial backing to the thought that John Whitney of Gorsington, head of the Clifford branch, was indeed John Whitney, second son of Robert Whitney of Whitney. See the Origin section for details. I'm interested in your feedback.
Could you check out Family:Whitney, Isaac (1770-1818)? According to Pierce, the son James was born in Middlesex Co., NY and d.s.p. 1847. However, he is currently linked to a James Watkins Whitney, b. Italy, NY, son of Fisher Whitney. This doesn't appear to be a correct identification.
Never mind - I've figured it out. On p. 241 of Pierce, James Watkins Whitney was given the number 1713 in error. He should be number 1714. An error in 1895....
I wanted to give some background on a few upgrades to the website that I plan to make:
Upgrade from MediaWiki version 1.5.4 (outdated) to 1.5.8- COMPLETE
- Upgrade from MediaWiki v1.5.8 to v1.6.8 (major underlying database change, first attempt at this ran into problems, attempting to work out now)
- Upgrade from MediaWiki v1.6.8 to v1.7.1 - May or may not do this at this time. Might give us additional features, but the 1.7 branch is fairly new and may still contain defects.
- Install new CreateArticle extension, which could allow us to implement templates for adding extracts for census families, etc.
Nothing that you need to do, other than to watch for and follow the red site notices at the top of the pages.
I have been working on some advanced capabilities to make it much easier for new users to create family group pages. Instead of dropping them into an editing form full of strange formatting characters, I envision presenting them with a form to fill out. They enter the information and then click a button to save it, and a properly formatted family page is generated.
Although it is still far from polished, I do have a working prototype. Take a look at User:Tdoyle/Sandbox3 and try filling it out. When you click the Submit button, you are then shown an edit screen with all of the data you entered filled in. In this way you can see how it works without saving dummy pages on the site. If you look at the contents of User:Tdoyle/Sandbox3, you will see that it is simply an HTML style form, which we can format any way we'd like. When the submit button is selected, it sends the information to a new page/routine called Special:CreateArticle, the code for which I installed yesterday. This code loads up a template, in this case Template:Family Starter, and simply plugs the information in. We can adjust Template:Family Starter in any way we'd like as well. We can use this same feature to allow users to input other data as well, such as census extracts, etc. I am going to continue to tweak and improve this, but would be interested in any comments you might have.
Blank White page using form creation page
That's what happens when a program ends unexpectedly - it doesn't return any results whatsoever. Hmmm... something obviously wrong there. I just tried it and it worked fine. Try going back to the page, pressing Ctrl-F5 to reload the page (to make sure you have the latest), then trying again. Try with just minimal information entered at first. Please let me know what you find.
Glad to hear it worked! I've corrected Sandbox3 so it now includes the Family: prefix. As far as the question marks go, I will perhaps be able to enter such defaults, but I'm not sure how yet - I'll havw to think about that. My goal here is to provide a method so novice users can enter their family data and create pages which are 90% correctly formatted. Advanced users (you, me, Andaleen) would then go in and properly link in the ancestors, provide general cleanup, etc. We can make improvements along the way, but I know we'll never be 100%
Ah - I understand and agree completely!
Isaac Amsden Whitney
Archive:WRG Photo 15 shows a photograph of Isaac Amsden Whitney with the lineage stated as Isaac Amsden6 Whitney (Capt. Jonathan5, Jonathan4, Jonathan3, Benjamin2, John1). However, Family:Whitney, Isaac Amsden (1805-1876) states his lineage as Isaac Amsden7 Whitney (Ami6, Jonathan5, Jonathan4, Jonathan3, Benjamin2, John1). Are these two different people? Was there a mistake made in adding the lineage to the photograph? Was some new piece of data known that corrects Pierce's stated lineage?
Also, I noticed that although Isaac was linked to his father Ami's page, Ami's page was not linked to his. Ami is in the 6th generation and his page was created by Andaleen. Perhaps we missed similar cases when weaving together the families.
Trip to Boston
I am scheduled to go to Boston from 8/3 to 8/7 to attend the Wikimania 2006 Conference, an annual event related to MediaWiki technology. I am contemplating extending my stay for a day or two to visit the NEHGS. If I were to do this, is there anything pressing that you would like for me to obtain while there?
Thank you for the "to-do" list. That will really help me to be productive in Boston!
I have started a small little project for fun which you, being a retired mathematician, may find interesting (or you may just tell me that I am way off on this one!) Take a look at How many Whitneys? I have extracted counts of Whitneys in census records as well as by meticulously counting the number of individuals in the early U.S. history through our family pages. At some point this manual counting method will stop and hopefully we'll have some mathematical way of determining approximate counts. My numbers in red are based upon very rudimentary mathematics, so please feel free to adjust. These numbers may help us in determining how 'complete' our family group pages actually are. Comments welcome.
It looks as though we're both editing this family. You just saved [[Family:Whitney, James (s1745-a1830)]] about the same time that I moved the page to Family:Whitney, James (1745-1830). It appears we have two pages now.
How Many Whitneys
I took the existing data from the count of how many Whitneys there were in the early years (based upon known family growth data) as well as from later years (from census information) and fed it into a statistical tool called Minitab. Using this, I was able to find a fitted line using quadratic regression using the logten of the population numbers. This resulted in a formula which fits the known data fairly well - S=0.0359676 and R-Sq(adj) of 99.9% Using this formula, I have plotted the known population data in blue and the statistical data in red. I have provided both a standard graph as well as a logarithmic scale graph to demonstrate how well this predictive model fits the known population data. Unless you see a glaring example of what I have done here, I'll add the predicted population numbers to my table and mark them as such.
Thank you. I am now trying to adjust my formula to use the actual year instead of an index number (1=1640, etc.) so that it makes a little more sense, and I am having what I believe to be precision issues in the formula that Minitab is reporting to me. I have contacted my Six Sigma Black Belt instructor who happens to be a statistician to help with this. Once I have this, I plan to post the formula on my How many Whitneys? page.
I have thought about the 1790 exercise. I want to do two things with this project. First, Make sure our indexes are correct, have all spelling variants, include the population counts, etc. Then I want to link to the families so we can verify we have everyone in both places. This was exactly what I started in Rhode Island which led to the two new family groups yesterday. If that keeps happening, it'll be a slow progressing project! I am planning to do this with the Ancestry databases, but if you want to do it with HQ as well, that'll only improve the accuracy. I'm not sure it's worth the duplicated effort though, especially with as many other projects as you currently have going.
As far as the category changes go, if you have a pattern such as "Castle, New York" to "Seneca Castle, New York", I can easily do this with the bot. I do not yet have code written to do more complex things, such as adding a second category tag for the county, state. Let me know if you'd like the bot to be put to use. For example, I can easily change all of the 1.'s to *'s in the reference.
African American Whitneys
Yes, I figured that out shortly after I wrote it as I started to trace Belden Whitney through the census records back to 1830, but I haven't changed the wording yet. I'm not sure what the answers are to your other questions. This is a new area for me and I am learning as I go. With 3.5% of all US Whitneys in 1870 being African American, I thought it was time we at least started a page or two to help researchers in this area. I'm sure my current efforts will morph into something much better over time.
New Family Creation Page
I've been working on my new extension to create a Family Group page. I now have the issue of the long URLS fixed, and am set up to be able to fine-tune the output based upon data not being entered in certain fields. I also still have to change a few things for security reasons. I've enhanced the formatting of the input page as well. Take a look at Create a Family Group Page and let me know what you think.
Good to have you back - I was beginning to wonder if you had gone on vacation or had been sick.
Location Categories, IV
Excellent to hear that you've finished the categories project! Great work! As far as the Canadian counties, I can't say that I am leaning either way. I've done no Canadian research, so I really don't know if this is common or not. I think the province name is sufficient, but perhaps we should ask some Canadian researchers for their opinions here.
I've noticed that my new Create a Family Group Page is working, but has issues as well. I've managed to lock down the security issue with allowing raw html on any page (we can only add it to the MediaWiki: namespace now, and only sysops can edit this namespace. The save routine is currently saving with the user's IP address instead of their username, and is allowing saves by people who are not logged in. This I need to fix quickly. Once I get that done, I plan to add additional logic to ensure basic information fields (subject first & last names) are entered, and will try to clean up the output when certain fields are not populated.
Once we get this working properly, we can then add additional entry forms. Imagine standardized entry forms for census extracts, etc.
Master Place List
If I remember correctly, we're going to use that list for the bot to verify / update the location categories. Since that list was originally generated from your original Pierce place list, and because most (all?) of your hand edits were to correct problems with the original script, I would think that most of your changes would not need to be added to this page. I would review it to verify that the before / after examples there are what you would want the bot to do.
Thanks for the comment about the trip. I leave tomorrow for Boston, have the Wiki conference Fri-Sun, and will be doing research Mon & Tues, returning Wed. I'll have my laptop with me and the hotel has high-speed access, so I should be able to continue to monitor the WRG site. My partner will be here at the house and can help out if we have any larger issues (such as restarting the server in the event of a power outage).
I updated my Create a Family Group Page routines this morning to 1) prevent new pages from being saved if a user is not logged in, 2) Ensure the subject's first and last name fields are populated, and 3) Set the birth/death year fields to "?" if they are left blank. I am sure we'll have more such customizations as time goes on.
: Family:Whitney, Henry M. (1843-?
Hello Robert -
I took a look and YES I recall discussion that Josiah Dwight is a cousin so I suspect you are correct about this. i am working on my family to get them to write it all down for me and then we can put it in here.
Already, putting the info up online has produced good results, as someone who was trying to get in touch with us has been able to make contact.
Is there a format with which I can submit to you which would be useful in filling in blanks?
Removing a User
The basic answer to this is "do not delete users". This causes issues with the underlying database, especially if these users have edited or added any pages. The software does not support this action, and everything I have read says don't do it. One capability that I can add faily easily is the ability to rename accounts. If someone initially registered with thei full email address, for example, and didn't realize how public it would be, I could change that for them. I'll have to install a new extension to make thia happen.
Having a good time here in Boston at the wiki conference. I'll be able to start my researh Monday.
I am here at the NEHGS Library in Boston. I copied the Paxton & Plymouth vitals (though it looks like we already had them extracted). No Whitneys in Raynham. The 1798 Direct Tax is online, so I'll do that when I get home. I photocopied ME records for Hancock, Penobscot (none), Washington (none), and York. I also went through the list of ME probate records you gave me. I have not done any military records.
I spoke with Gary Boyd Roberts. Did you know he mentioned your Thomas Whitney theory in a book he recently published?
Create a Family Form
The form is currently in a state where it works, but it is fairly 'dumb'. It takes the information you entered and just replaces it into the template and creates the page. This is why you are seeing ten children (no more, no less), the gender is fixed, etc. Think of this as a "working proof-of-concept". Even with these limitations, I think it's now much easier to add a new family than it was previously, but yes, the results do need tweaking after creation.
How can we improve it? Well, here are some ideas:
- Male/Female checkbox which would alter the pronouns in the template. Need to create a systematic way to mark words to be changed though.
- Implement some method to limit the number of children output.
- Checkbox to include child? What if you forgot to check it - all data would be lost.
- Check all fields for input data - if any (other than surname) are filled in, output the child.
- Ask the user how many children in the family and dynamically modify the input screen - much more difficult to implement.
All of these will require me to dedicate some brainstorming and programming time to come up with solutions. These are all ideas that I have been thinking about, but it's a matter of prioritization.
I'm not sure if you saw, but I started to transcribe the Hancock Co. vitals this morning. Not sure if there's anything new in there or not though.
Recent Changes Omission
I haven't figured out exactly why things sometimes don't appear in the Recent Changes page, but I have noticed that as well. Take a look at the Special:Log page and you should see all of the changes that are things other than edits.
Polly (Holland) Whitney
I don't know if I am doing this right - here is a reply to a message you left me. I viewed this after submitting and had to add the contact info after the fact.
On 23 June 2006 you left a message about Mary "Polly" Holland a descendant of John Howland, "Mayflower" passenger. I have looked at the archive and would like additional details. As you can see it has been over 2 months since I have logged in - I don't spend much time lately on family history, but the interest is still there. I'm hesitant to post my database since it is mostly a collection from others and it has not been verified. I'm at a bit of a loss as to how to clean it up - it seems like an insurmountable task.
Robert: I just sent you an email re: Reed.
Check your email for a message I sent regarding an exciting discovery...
This is a very curious case indeed! I have not responded until now because I did not have a good answer, and I'm still not sure that I have one. My suggestion would be to create three pages, one for each of the Whitney descendants. On these pages, list the relative information, children, marriages, etc. I would then add a note indicating that husband Moses Blood also had other wives, and list the links to their pages as well. Since our primary objective is to list the families from the Whitney perspective, I would not feel obligated to provide a page with all of the particulars of Moses Blood.
What should we do with this page, created by another member: Jonathan Whitney Will?
5th Generation Done - Next Projects
Congratulations on finishing the 5th generation! That was a lot of work for both you and Andaleen and it really makes our website all the better!
Here are a few ideas that I have had for additional projects:
- Medieval / early English Whitney families
- Research immigrant families to other countries. I've recently done a bot of searching and have made contact with an Australian Whtiney family. I also found Whitneys in South Africa.
- Create archive pages for the military extracts that Ken Whitney has been posting to the mailing list.
- Trace families from key locations in England from the 1841 census down through the last available census in an attempt to locate living male line descendants who could join our DNA study.
Clifford and Whitney MSS No. 38
I found this on the web this morning: http://freepages.genealogy.rootsweb.com/~roseana/lease1614.htm