Restrictions: Redistribution and use of this manual in physical and binary formats, with or without modification, are allowed under all conditions. This manual also does not cover the procurement of a valid license for this software




Yüklə 113.05 Kb.
səhifə1/3
tarix28.02.2016
ölçüsü113.05 Kb.
  1   2   3



The BEAP Process


Instruction Guide


Manual Version 1.6 – 2/17/03

Prepared by: Philip K Ng




Restrictions:
Redistribution and use of this manual in physical and binary formats, with or without modification, are allowed under all conditions. This manual also does not cover the procurement of a valid license for this software. The user of this manual is expected to purchase a valid serial key from the software producer.

Warranties:
This manual is provided by the author "as is" and all expressed or implied warranties are disclaimed. In no event shall the author be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to loss of use, data, profits, or business interruption) arising in any way out of the use of this manual.

Table of Contents

Restrictions …….…………………………………………………………………………Pg 1
Warranties .………………………………………………………………………………Pg 1
Section 1 – Introduction ……………………………………………………………….Pg 3

1.1 – User Expectations and Expected Training ……………………………….Pg 3


1.2 – Applicability ……………………………………………………………….Pg 3
1.3 – Purpose and Usage Description ……………………………………………….Pg 3
1.4 – Conventions ……………………………………………………………….Pg 3
1.5 – Contact ……………………………………………………………………….Pg 3
Section 2 – Instructions and Guide ……………………………………………………….Pg 4
2.1 – Installing X-Win32 ……………………………………………………….Pg 4
2.2 - Using X-Win32 to run GUI programs on a remote UNIX workstation ……….Pg 5
Section 3 – Known Error Messages, Known Problems and Error Recovery ……….Pg 7

3.1 – Cannot Find Display ……………………………………………………….Pg 7


3.2 – X11 Tunneling not enabled ……………………………………………….Pg 8
Section 4 – Bibliography ……...……………………………………………….……….Pg 9
Section 5 – Glossary ……………………………………………………………………….Pg 9

Section 1 - Introduction
1.1 - Experience and Training

This manual was written for the database managers of the BEAP database. This manual assumes that the database managers have knowledge in the following PhilSoft software utilities: BEAP File Converter, BEAP SPSS Automater Utility, BEAP File Mover, and the BEAP MassMailer Utility This manual also assumes that the user is familiar with the use of the Microsoft Windows Operating System. A familiarity with the following software packages is desired but not required: SPSS, Microsoft Outlook, and Internet Explorer.


1.2 - Applicability

This document covers installation and basic use of the software on a computer of the following minimum specifications:



  • Intel Pentium 2 MMX running at 200 MHz

  • 128 Mb of System RAM memory

  • 10 GB Hard Drive

  • Windows NT4 or better

In addition, the BEAP process requires facilities to “op-scan” the forms, and return such data as a column delimited data file.



1.3 – Purpose and Usage Description

The PhilSoft software packages were written to assist in the complex data-crunching that was required to process each submission to the BEAP database. This manual was written to guide the user through the conversion of a single submission to the BEAP database. This manual was also written to guide a user through a “rehashing” or reprocessing of the BEAP database.


1.4 - Conventions

Illustrations, with figure numbers and captions immediately below in italics, show the screens before each action. Important text will be highlighted in a bold font. Links, such as email addresses or web addresses will be in dark grey underlined font. Notes and important information will be shown with bullets inset from the text. If a program is used and its name is shortened in the subsequent text, the program will be named and the abbreviation will be placed in capital letters immediately to right of the term.


1.5 – Improvement

The author of this manual welcomes any comments, questions or complaints to further improve this and other documents. You can reach him by sending an email to pkn1587@cs.rit.edu.



Section 2 – Instruction Guide
Section 2.1 – Sending a file to the processing center
This section covers the conversion of one single file through the BEAP process, from start to end.


  1. The submission is received in the BEAP processing facility through postal mail.



  2. The submission is preprocessed for potential errors. The data on the monthly report (facesheet) is compared to the existing database for errors in reporting. If the program already exists in the database, the program’s contact information is checked first. This includes the program’s contact person and the contact person’s email address. Next, the number of graduates is checked against the existing data. If the number does not correspond with the existing data, the submission should be flagged for data verification.



  3. The submission’s forms will be verified against the monthly coversheet. Each form is counted, and checked against the numbers entered on the monthly report. If any of the form counts do not correspond to the monthly coversheet, the submission should be flagged for data verification.



  4. The submission is now clear of major processing errors. For each set of forms (consisting of forms of the same month, year and type). The following process must be applied.



    1. The forms will be separated if necessary.

    2. Place a “post-it” or sticky note on the top of the “front” of the set of forms. On the note the following data will be written: filename, type, form count, and date.

      The filename consists of the following:



      1. First 4 characters = Proganym (If program is new to BEAP, a unique one needs to be created from its university or college initials. Otherwise use the one previously created.)

      2. Characters 5 & 6 = EN, VN, EX, VX, AL, or EM (depending on the BEAP instrument being submitted) Character 7 = Month indicated on facesheet (1-9=Jan-Sep, R=Oct, S=Nov, T=Dec, A=rescan)

      3. Character 8 = Year indicated on facesheet (0, 1, 2)

      4. Character 9-12 – “.dat”



    3. The remainder of the note consists of the ”SCANTOOLS PROGRAM" and "#". These are to be selected from the following choices: (entrance #901, values #902 (Use with either SWVI form), exit #903, alumni #904, employer #905)

      See Figure 2 for a sample note.



      Figure 2 – Sample post-it note. In this sample, RITN is the proganym. This is a submission of Exit data done during May 2003. There were 13 forms in this submission. The “thanks!! –Phil” is not necessary (but is a nice touch).



  5. Place in an interoffice envelope, or in a box (8 1/2" x 11" x ?') and address to:

    Debi Fitts, ITS Data Center Operations, A-367 Ross



  6. Send the envelope or box of submissions via interoffice mail.



  7. Enter facesheet data into the BEAP Facesheet Data Excel spreadsheet. For every set of forms submitted in one month (all forms collected in April of 2001 are considered a “set”), a new row must be created in the facesheet spreadsheet. The monthly report data should be entered in the following order:

    FULLNAME, PROGANYM, DATE, TODAY’S DATE, PROGID, PROGTYPE, STATE, CSWEREG, BPDREG, MAXGPA, AUSPICES, CANDIDCY, ACCREDIT, NUMGRADS, EN-OUT, EN-BACK, VN-OUT, VN-BACK, EX-OUT, EX-BACK, VX-OUT, VX-BACK, AL-OUT, AL-BACK, EM-OUT, EM-BACK



    For all data entered after “NUMGRADS”, the cells which contain data should be highlighted in light green to signify that they are sent to scanning. See Figure 3 for an example.




Figure 3 – Facesheet spreadsheet shown with green highlighting. The key is located on the upper left hand corner of the screen, under “color code”.



  1. Save Excel spreadsheet and increment version number after an input batch is finished.

  2. Repeat steps 2 through 8 to process more data for scanning.

  3. Save Excel spreadsheet and increment version number after an input batch is finished.

  4. Close the Excel Spreadsheet.


Section 2.2 – Processing of BEAP files returned from data processing center
This section covers the processing of the files received from data processing. After we send the processed and labeled forms to data processing, the data processing center staff will scan the forms and upload the files to the VAXC account. The staff will send us an e-mail, stating that the forms have been scanned, and will return the scanned forms via interoffice mail. After we receive an e-mail from the processing center, we can proceed to proceed to process the files. The process is detailed below.
Part 1 – Fetching scanned data from the data processing center


  1. An e-mail from the data processing center is received.

  2. Open the telnet program, via Start Menu>Run>>”telnet vaxc.rit.edu”

  3. Log onto vaxc.rit.edu as the username “694dept”.

  4. Change directory to beap by typing the command “beap”.

  5. Create/or access the folder structure in the following structure: year, month, date. This is to keep a backup of the received scanned data. See Figure 4 for an example of this structure.


    Figure 4 – Naming Structure of the backup folders on the VAX account



  6. Enter the command “receive *.*;* *.*;*” to receive all of the files uploaded by the data processing center staff.

  7. Then type “ex” command to leave receive mode, and “lo” command to log off the server.


Part 2 – Receiving data from the VAXC server
After receiving the files from the data processing center, they can be transferred using ftp for processing on the BEAP database manager’s machine. The following outlines this process.


  1. Run WS-FTP to transfer files to the database processing machine (the machine equipped with the PhilSoft programs listed in the introduction, section 1.1 ).

  2. Use ftp to connect to “rit.edu”. The username is 694dept.

  3. Navigate to the location which the files were received in section 2.2 part 1 step 5.

  4. Using WS-FTP, transfer the scanned files to the incoming scanned data folder of the BEAP database. This is by default “e:\1data\beap\1scanned”.

  5. Log off WS-FTP by closing the program.


Part 3 – Data Processing


  1. Open the Excel facesheet database.

  2. Save the facesheet database as a “facesheetXX.CSV” (where XX is the current version of the facesheet) file so that the PhilSoft conversion utilities can operate with less user input.

  3. Close Excel to prevent making changes to the CSV file as opposed to the facesheet database.

  4. Reopen the Excel facesheet database (which was opened in step 1). Make sure that the file opened is not the csv file.

  5. Open the BEAP Native DAT file converter (CONVERTER). If the “Tell me when there are newer files in the BEAP Facesheet folder” option is selected in the “Start/Facesheet” pane in the “Settings” pane of the CONVERTER, it will inform you if there are newer files in the folder which the CSV is located. See figure 4 for the message that is displayed when there are newer files in the CSV directory.

    Figure 4 – Message displayed when then “Tell me when there are newer files in the BEAP Facesheet folder” option is selected and there are newer files in the CSV’s directory.



  6. Change the CSV file location of the converter to match the file that was chosen on step 2 above.



  7. Make sure that the settings in the setting panel of the CONVERTER are correct. This includes the correct converted DAT file output file location, database location, scanned file moving location, facesheet CSV file location, etc. See Appendix Section 1 for a detailed explanation of all of the converter’s settings.



  8. On the left hand side of the CONVERTER program, locate the DAT files which you wish to convert. After locating the files you wish to convert, click on the desired file to select it. The converter will load the file, and a blue screen will show the expected results from conversion. If the file was named and entered into the facesheet database correctly before sending the file to the data processing center (Section 2.1), the end of each line will be appended with this file’s facesheet data (See figure 5) . If the data that is appended reads “XL DATA HERE >|”, then the CSV file representing the facesheet spreadsheet has not been loaded or the file has either been incorrectly named or entered in the facesheet spreadsheet (See Section 3.2 for possible resolution).

    Figure 6 – The results of loading a file which has been correctly named and entered in the facesheet spreadsheet. The circle (1) shows the vital statistics of the file, such as the filename and the number of forms it has found in the file. The circle (2) shows the data that it has extracted from the facesheet.(note: the asterisks will become spaces after conversion.)



  9. Locate the same entry in the BEAP facesheet. It will be highlighted in green. (See figure 7)

    Figure 7 – The entry which corresponds with the file to be converted above (with the entry highlighted in green)



  10. Press on the “fill box with color” button in Excel, and select yellow. (See figure 8) This indicates that the file has been converted, and is available to you for future reference.



    Figure 8 – Selection of yellow to mark completion of an entry in the Facesheet database



  11. Save the facesheet database.

  12. Switch to the converter and press the “Save Conversions” button located near the top center. This will convert the file, append the facesheet data, and perform the functions selected in the “Settings” panel of the BEAP Native DAT File Converter.

  13. You may repeat Steps 7-11 to convert more DAT files.


Part 4 – Data Processing
Note: This section only applies if the user has set the converter to automatically create SPSS syntax files. If the user has chosen not to have the SPSS syntax files automatically created, the user must manually create the syntax files before proceeding.


  1. Start SPSS.

  2. Run the “Automater” (AUTOMATER) program to convert the files to reports and perform the residual data processing tasks. See Appendix Section 2 for a detailed explanation of all of the AUTOMATER’s settings. After running the AUTOMATER program to process the files, there will be several html files in the output directory that was specified in the Automater’s settings.

  3. Run the “HTML Purifier, File Mover, and Compressor” (FILEMOVER) program to copy the html files to their respective directories in the BEAP database, move them to the folder structure of the web server, remove HTML tags that cause errors in Netscape browsers, and compress them to save space on the web server. After this process, the output directory that was specified in the AUTOMATER’s settings will contain several directories, each html output file will be filed into a directory named with the first four letters of the filename (the proganym).If the “compress the files to save space on the web server” option was set in the FILEMOVER, the file will be compressed and will have the extension “.htm.gz”.

  4. Run WS-FTP to transfer files from the database processing machine to the web server. To upload, first locate the “users” directory in the “www” folder of the BEAP account on the right hand pane. Then in the left hand pane, navigate to the output directory that was set in AUTOMATER, select all of the directories on the left and pane and then press the upload button, or the button labeled “->”.

  5. Using WS-FTP, navigate to “cgi-bin” directory in “www” folder of the BEAP account in the right hand pane. Afterwards, navigate to where the CSV file is located that was generated in Section 2.2, Part 3, Step 2, and upload that file to the web server.

  6. Run Internet Explorer to navigate to the BEAP website. Login to the BEAP website using the administrator account. On the BEAP administrative site, locate the “Import CSV File as Database Source” section. Enter your administrative password again, and the filename of the CSV file that was generated in Section 2.2, Part 3, Step 2. After entering this data, press the “Press Here to Continue” to load the facesheet’s updated information to the web site. See Figure 5 for an illustration of the BEAP website administration page, and Figure 6 for an illustration of the resulting page generated.

    Figure5 – BEAP administration page showing CSV file importing.

    Figure 6 – BEAP Administration page showing successful CSV facesheet file loaded on the web server.



  7. Run the “Mass Mailer Utility”, and then press the pink “Start” button to send an email message to all of BEAP users who have new files that have just been processed. See Appendix Section 4 for a detailed explanation and illustration of the “Mass Mailer Utility’s” settings.

  8. Using WS-FTP, navigate to the “cgi-bin” directory in the “www” directory of the BEAP account, and delete the CSV facesheet file on the web server.

  9. Erase all the directories and files in the following directories:

    1. The SPSS “ready for processing” folder that was setup in the CONVERTER program (1spssready)

    2. The output folder that was set up in the AUTOMATER’s settings (1report)



After completing all of these steps, you have finished processing data.
Section 2.3 – How to Recompile the Database
Because data in the BEAP database is constantly changing, data on the facesheet occasionally changes, and over time, the reports will no longer be consistent with the corresponding entries in the database. Also occasionally, the syntaxes to generate the reports changes and it is appropriate to regenerate the reports. This process of regenerating reports to match the latest changes in the database is known as “recompiling” the database. To perform this operation, please follow the steps below.


  1. Start the CONVERTER

  2. Make sure that the settings in the setting panel of the CONVERTER are correct. This includes the correct converted DAT file output file location, database location, scanned file moving location, facesheet CSV file location, etc. See Appendix Section 1 for a detailed explanation of all of the converter’s settings.

  3. Check to make sure that a valid and current facesheet CSV file is loaded, and that the settings for creating SPSS files is correctly set up. Refer to Appendix Section 2.1, Part 2.3 for detailed instructions and explanations of how to set up the SPSS syntax generation feature.

  4. Click on the “Settings / Other” tab of the Converter.

  5. Click on the “Recompile” sub-panel of the “Settings/Other” tab.

  6. Navigate to the BEAP database folder using the three left hand fields. You only need to go to the higher level folder. See Figure 7 for an illustration.


    Figure 7 – The converter after navigating to the BEAP database folder. Note that there are no files listed in the lower file list box.


  7. Press the “Start Auto-Recompile” button. This will process every subdirectory in the BEAP database and will convert every raw data file according to the settings of the converter. If an error is encountered during the conversion process, it will be ignored and it will continue with the next file.

  8. Perform the “Data Processing” steps found in Section 2.2 Part 4.


Section 2.4 – How to Generate Total School and Data files


  1. Start the “BEAP Totalizer Utility.” (TOTALIZER)

  2. Ensure that the settings are correct. For a detailed explanation and illustration of these settings, please refer to Appendix Section 4

  3. Press the pink “Start” button.

  4. Perform the “Data Processing” steps found in Section 2.2 Part 4.

Section 3 – Known Error Messages, Known Problems and Error Recovery
Section 3.1 – Data Verification Process
This section describes the process which must occur if there are errors on the monthly coversheet which was provided to us. This process should occur well before processing (splitting and labeling) of the forms described in section 2.1.
A facesheet is considered “needs data verification” or “bugged” if the facesheet is:

  • The facesheet is missing entirely.

  • Number of graduates reported to CSWE is missing.

  • Number of graduates reported to CSWE in that time frame does not correspond to the number reported in previous submissions for that time frame. All submissions between November of one year and November of the following year is considered in the same time frame. If the user submits a facesheet in November 2000 with the number of graduates reported in November of 2000 as being 10, and in December 2000 submits another set of data with the number of graduates reported to CSWE in November 2000 as 15, then the data is requires verification.

  • Program data such as maximum GPA attainable, program type, or program auspices is missing and cannot be ascertained from previous submissions in the database.

For each of the situations listed above, the problem can be solved by sending an email or by calling the program in need of data verification, and asking for the data required. If the facesheet it missing, the program should be asked to fill one out and have it faxed or mailed to us.




Section 3.2 – ‘XL DATA HERE > |’ is appended to the end of each line of a data file after selecting a file in the BEAP Native DAT file Converter.

This section describes the process which must occur if the BEAP Native DAT File Converter cannot find the appropriate facesheet data string to append to the end of each file. Such errors would occur in Section 2.2, Part 3, Item 7.


Possible solutions are listed below.

  • The facesheet has not been set or loaded. Set the facesheet CSV location in the “Start/Facesheet” pane in the “Settings” pane of the DAT File Converter.

  • Facesheet lookup has been disabled in the settings screen. Enable it in the “Start/Facesheet” pane in the “Settings” pane of the DAT File Converter, or manually copy the data from the Facesheet spreadsheet and follow the screen prompts to complete manual conversion.

  • The facesheet CSV file is an older version, and this file’s data is not in it. Close the Converter and follow Section 2.2 Part 3 to restart the conversion process.

  • The file was not correctly named, or was not correctly entered in the facesheet spreadsheet. Check the spreadsheet and filename against the facesheet that was received and adjust accordingly.

To restart the conversion process on this file, one can follow the steps detailed in Section 2.2 Part 3 or use the “refresh” button located on the lower left corner of the converter, or press the “Apply and Reload Facesheet” button located in the “Start/Facesheet” pane in the “Settings” pane of the Converter.



Section 4 – Appendices
Section 1 – Converter’s Settings

Part 1: - Startup / Welcome / Common Screen



  1. This selects what drive volume the current file listing (2) will show.

  2. This selects what files the current directory (3) will show.

  3. This selects what file to open. As soon as one clicks on a file in this directory, the converter will open the file that is selected, and attempt to convert it. If it is successful, then the conversion will be shown in the “Expected Conversion” tab (9), and the original file will be displayed in the “Original File” tab (8).

  4. This is the Save Conversions button. If a file is selected, and there is data in the “Expected Conversion” tab (9), and the facesheet data has been looked up, (assuming the option is chosen in the “Settings” tab (Part 2.1, Item 3 - 5), then the file will be converted and saved to the settings in the “Output Settings” tab (Part 2.2 all). If the facesheet is not loaded, or the file does not match the facesheet’s data, or facesheet lookup is disabled, a box will appear, requesting the Excel spreadsheet data. To manually look up the facesheet data, find the line in the spreadsheet that corresponds to the file, and copy the fields “PROGID” through to XX-BACK, where XX is the file type. Place this copied data in the input box, and remove the items which are after the number of graduates and before the last two items separated by the “|” character. The resulting string end with ”XXX|XXX|XXX” where X is a number. See Figure A1 for an illustration of this.



Figure A1 – finished Excel spreadsheet data that is correctly entered.



  1. This area displays file information such as the number of forms in the file as well as its name. This will always remain on the top after resizing the screen.

  2. This button refreshes the file listing box (3).

  3. This button renames a file selected in the file listing box. To use this feature,

    1. Select a file in the file listing box (3).

    2. Press this button.

    3. An input box will appear asking for the desired resulting name.

      1. If the file cannot be renamed or no file is selected, an error message will appear.

  4. This is the panel that the original contents of an unconverted file will be shown. (Note that this is a read-only text panel, and changing it has no effect on the resulting file.)

  5. This is the panel shows the expected conversion contents of an unconverted file. This conversion process concatenates forms with the same ID numbers, stripping out the form numbers. This data is then removed of all asterisks, and if the option is set, the excel spreadsheet data is appended on the end of each line. (Note: that this is a read-only text panel, and changing it has no effect on the resulting file.)

  6. This panel shows the results of the file conversion. The asterisk stripping operation is performed again, if the Excel spreadsheet data contains asterisks. (Note: that this is a read-only text panel, and changing it has no effect on the resulting file.)

  7. This is the settings panel, and is described in detail in part 2.

Part 2.1 – Settings Sub Panel





  1. Our mascot.

  2. If you enable this option, the converter will ignore all of the settings that you have set in the BEAP Facesheet Settings frame (3, 4, 5), and will ask you for the excel spreadsheet string that you would like to append to the end of each line in every file. This setting will is not remembered and will be reset each time the program is loaded.

  3. This is the facesheet CSV location. After a CSV file is loaded or reloaded, the read only text field immediately below the filename will display the last modified date of the file. If the CSV file cannot be loaded or the file is not specified, the information field will display “CSV file not found or cannot be loaded.” To select the CSV file using a graphical method, press the “..” button to display a Windows file selection box. Please note that this setting is common among all of the BEAP processing programs and changing it will affect them.

  4. This button is used to apply the changes to the CSV filename field, (re)load the CSV file, and save the settings to the registry.

  5. This option, if selected, this program, on startup, will inspect the directory where the CSV file specified, and determine if there are any files that have been updated more recently then the timestamp that is on the CSV file. If there are such files, it will display a message for each file. (This setting will be applied and saved as soon as it is changed.)

  6. This field displays the directory that the converter will start up in default. This folder will be loaded in the “drive selection”, “folder selection list” and “file display list” (Appendix Section 1, Part 1, items 1, 2, 3.)

Part 2.2 – Output Management Sub Panel






  1. There are two major output modes that this converter is in. The first mode that the converter will be in is the “output to a folder” mode. The converter is in this mode if the green button that the label (1) is on shows “To Database” and the lower 3 text fields are grayed out.

    In “To a Folder” mode, the converter will:



    1. Convert the files, append the ending “–c.dat” to the file and save them in the directory that is specified in the “Data Output To.” The original files will be left alone.

      1. If the field labeled “Data Output To” is empty, the files will be saved in the directory that this application is stored.

      2. If the folder does not exist, an error message will be displayed.

      3. If you wish to graphically select this directory, you can use the “..” button to display a Windows Folder Select window.



In “To Database” mode, the converter will:

  1. Convert the file, append the ending “-c.dat” to the end of the file and save the converted files to the location specified in the “Converted File To:” field. It will also by default, move the original scanned file to the location specified by the “Move Originals To:” field (where @Proganym@ refers to the first four letters of the file).

      1. If the locations specified by either of the “Converted File To:” or “Move Originals To:” field do not exist, the respective directories will be created. If they cannot be created, the converter will display an error message and exit without converting the file selected.

    1. To setup the “To Database” mode correctly, after pressing the “To Database” button, a wizard will appear to assist you by asking several questions. These folders will be added to the “Folder Creation” section (Appendix Section 1, Part 2.7)




  1. If this option is set, the unconverted scanned data file will be copied to the “Move Originals To” field as opposed to moving it. The file will remain after converting it. Note that this setting has no effect unless the converter is in the “To Database” mode. (This setting will be applied and saved as soon as it is changed.)

  2. No settings will be applied or saved unless this button is pressed.



  1   2   3


Verilənlər bazası müəlliflik hüququ ilə müdafiə olunur ©azrefs.org 2016
rəhbərliyinə müraciət

    Ana səhifə