User manual SAS TEXT MINER 4.2 GETTING STARTED

[. . . ] Getting Started with SAS Text Miner 4. 2 ® SAS® Documentation The correct bibliographic citation for this manual is as follows: SAS Institute Inc. Getting Started with SAS ® Text Miner 4. 2 Copyright © 2010, SAS Institute Inc. , Cary, NC, USA ISBN (electronic book) All rights reserved. For a hardcopy book: No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, or otherwise, without the prior written permission of the publisher, SAS Institute Inc. For a Web download or e-book:Your use of this publication shall be governed by the terms established by the vendor at the time you acquire this publication. [. . . ] Note that the misspellings patien, patietn, and patie are included as child terms. Create a Stop List A stop list is a simple collection of low-information or extraneous words that you want to remove from the text, which has been saved as a SAS data set. Make sure that the Freq label has an arrow that points downward to indicate that the Freq column is sorted in descending order. Hold down the CTRL key and click on these terms: patient, have, receive, vaccine, day, develop, and dose. This removes the checkmark from the Keep column for each term that you have selected. Create a Stop List 37 There are several more terms you could choose to exclude. If additional terms are dropped from the analysis, then your results will not match those later in this document. 3. Click the # DOCS column heading and ensure that the sort arrow is pointing upward. Right-click a selected term and select Toggle KEEP to drop these terms from the analysis. 38 Chapter 5 · Cleaning Up Text 5. Select the Mylib library and type VAEREXTSTOP in the Data Set Name box. Note that the Stop List property of the Text Miner -- CST node is set to MYLIB. VAEREXTSTOP. Explore Results Improvements 39 Explore Results Improvements You can redo clustering to explore the improvements to results from cleaning the SYMPTOM_TEXT variable. Verify that the Cluster property settings for the Text Miner -- CST node are the same as in previous examples. Click OK in the Run Status dialog box when the node has finished running. Select the button for the Interactive property in the Text Miner -- CST node property panel to open the Interactive Results window. Compare these results to the first Text Miner node results from "View Interactive Results" on page 21. Does the clustering seem to have improved with the cleaned SYMPTOM_TEXT data?Right-click on an empty space in the diagram workspace, and select Paste from the menu. Connect the Text Miner -- CST node to the Segment Profile (2) node. 8. Click the for the Variables property for the Segment Profile (2) node to open the Variables -- Prof2 window. Make sure that the PROB variables and the _SVD_ variables have a Use value of No. 9. Click OK to save the variables settings and close the Variables -- Prof2 window. Click Yes in the Confirmation dialog box. 40 Chapter 5 · Cleaning Up Text 11. Click Results in the Run Status dialog box to open the Results window when the node has finished running. Do the relationships appear clearer with the cleaned text than they did with the uncleaned text?Close the Results window. 41 Chapter 6 Predictive Modeling with Text Variables About the Tasks That You Will Perform . [. . . ] Use the Ignore properties of the Text Parsing node to limit parsing to high information words. You can do this by ignoring all parts of speech other than nouns, proper nouns, noun groups, and verbs. You can also use the Parse properties in the Text Miner node to ignore all parts of speech other than nouns, proper nouns, noun groups, and verbs. Structure sentences properly for best results, including correct grammar, punctuation, and capitalization. [. . . ]