linguistlist.org>
Editor for this issue: Ann Dizdar <annlinguistlist.org>
dcs.shef.ac.uk>Apologies if you receive this more than once. *********************** Also available at: http://www.dcs.shef.ac.uk/research/ilash/Meetings/SALT97 *********************** Speech and Language Technology (SALT) Club Workshop on Evaluation in Speech and Language Technology June 17-18, 1997 Halifax Hall, University of Sheffield, Sheffield, UK PROGRAMME *** June 17 *** 9:00 Welcome, Rob Gaizauskas, Sheffield 9:15-10:00 Maghi King, ISSCO and ETI The EAGLES Evaluation Group: Standards and Sharing 10:00-10:45 Lynette Hirschman, MITRE Language Understanding Evaluations: A Case Study of MUC and ATIS 10:45-11:15 COFFEE 11:15-1:00 Multi-site Evaluation Exercises Towards a methodology for evaluating information retrieval systems adapted to textual documents in the French language: the Amaryllis exploratory cycle A Coret, P Kremer, B Landi, D Schibler, L Schmitt - INIST AUPELF Project: Term and Semantic Relation Extraction Tools: Evaluation Paradigms C Jouis, W Elhadi - UFR IDIST/CREDO Industrial Parsing of Software Manuals: Empirical Qualitative Comparison of Parsers and Parsing Schemes E Atwell, R Sutcliffe - Universities of Leeds and Limerick Progress report on the GRACE evaluation program for French Part-Of-Speech taggers J Mariani, J Lecomte, P Paroubek, M Rajman, G Adda - CNRS 1:00-2:00 LUNCH 2:00-3:15 System Evaluation I: Spoken Dialogue Systems A Generic Template to evaluate integrated components in spoken dialogue systems G Churcher, E Atwell, C Souter - University of Leeds The DISC Project N Bernsen - Odense University Corpus-Based Test System for an Automated Speech Processing System L Rothkrantz, R van Vark, H Koppelaar - Delft University 3:15-3:45 COFFEE 3:45-5:00 Component Evaluation I: New Metrics + Tools Evaluation of word sense disambiguation programs: progress report A Kilgariff - University of Brighton Evaluation of discourse segmentation techniques and representations J Crowe - Harlequin Ltd. A method for prosody evaluation G Sonntag - University of Bonn 5:00-6:00 Poster Session 6:00 Sherry Reception 7:00 Dinner *** JUNE 18 *** 9:00-9:45 Steve Young, Cambridge Speech Recognition Evaluation: a Review of the ARPA Programme 9:45-10:45 System Evaluation II: Text Processing Systems Towards Small-Scale Evaluation of a Multilingual Information Retrieval Systems for Multimedia Documents K Krueger-Thielman - Universiataet Tuebingen Proposal for a user centred evaluation regime for automatic abstracting research F Johnson - Manchester Metropolitan University 10:45-11:15 COFEEE 11:15-1:00 Component Evaluation II: New Metrics + Tools Towards a Rating System for Speech Recognisers S Cox, P Linford, W Hill, R Johnston - University of East Anglia and BT The MUC Coreference Annotation: Status and Plans L Hirschman, P Robinson, J Burger, M Vilain A probabilistic error metric for segmentation algorithms D Beeferman - Carnegie Mellon University NL Module Evaluation in GATE R Gaizauskas, P Rodgers, H Cunningham - University of Sheffield 1:00-2:00 LUNCH 2:00-3:45 Perspectives on Evaluation: The Big Picture: Evaluation of Products, Projects, Programmes (SALT Evaluation and European Opportunities Mapping) N Ostler, Linguacubun Ltd. User-Centred Evaluation in Language Engineering J Tait - University of Sunderland Towards a standard reference system for Speech Technology evaluation D Johnston - BT Laboratories Evaluation in Verbmobil U Jost - University of Hamburg 3:45-4:15 COFFEE 4:15-5:15 Panel Session: Directions in S+L Evaluation 5:30 Closing Remarks *** Poster Session *** RQA methodology: towards a qualitative evaluation of speech understanding and spoken dialog systems. J Y Antoine, J Zeliger, J Caelen - VALORIA, ICP and CLIPS Subjective assessment methods for text-to-speech systems P Arden - BT Laboratories Tagging Email with 8 Tagsets: lessons on evaluation E Atwell, G Demetriou, J Hughes, M Schiffrin, C Souter, S Wilcox - University of Leeds A Tool for Qualitative Analysis of MUC-6 Performance P Callaghan - University of Durham A extendible specification and validation method for formal dialogue validation A Coillault - Universite Blaise Pascal Evaluating the Pronunciation Component of a Text-to-Speech System R I Damper, K. Gustafson - University of Southampton, KTH Sweden An annotated Reference Corpus as Diagnostic Evaluation Tool T Declerck and J Klein - DFKI Usability Evaluation of Speech Technology Applications C Delogu - Fondazione Ugo Bordoni Measuring the Uncertainty in Grammatical Classification G Demetriou, E Atwell, C Souter - University of Leeds Evaluation with entropy indicators C Lyon - University of Hertfordshire DiET : Diagnostic and Evaluation Tools for Natural Language Applications DFKI, Aerospatiale, IBM Germany, ISSCO, LRC, SRI International Evaluation of Translation Tools in Software Localisation A Rinsche - The Language Technology Centre Ltd. A Concatenation Framework for Text-to-Speech Synthesis using CVC Syllables R Sahandi, D Vine - Bournemouth University Usability evaluation of speech based sevices: The user perspective C K Sidhu - BT LaboratoriesMail to author|Respond to list|Read more issues|LINGUIST home page|Top of issue
irin.univ-nantes.fr>"Apologies if you receive this call more than once" - ------------------------------------------------------------------ Call For Participation RIAO'97 CONFERENCE Computer-Assisted Searching on the Internet June 25-27, 1997 McGill University, Montreal, Quebec, Canada http://www.sciences.univ-nantes.fr/RIAO97 [note: RIAO in CAPS] Brief Description : - ------------------- Every three years the Centre de Hautes Etudes Internationales d'Information Documentaire (CID) of Paris, France, along with various international affiliates, organizes an RIAO conference (RIAO is the French acronym for Computer-Assisted Information Retrieval) under the sponsorship of European Community (EC), Ministere des affaires etrangeres, Ministere de l'industrie, Secretariat d'etat a la recherche (France), Ministere de l'industrie, du commerce, de la science et de la technologie (Quebec) and CRIM (Canada). RIAO97 will be the fifth conference in the series. RIAO85 was held in Grenoble, France; RIAO88 at MIT; RIAO91 in Barcelona; and RIAO94 at Rockefeller University in New York. RIAO conferences have the special feature of incorporating both scientific papers and innovative product demonstrations. Both the product demonstrations and the scientific papers (which are often accompanied by prototype system demonstrations) are subject to a rigorous selection process. The mix of scientific expertise and state-of-the-art industrial development lends itself to a critical examination of both aspects, stimulating both new product development, ecnouraging sponsorship of start-ups, as well as initiating lines of further, critical research investigations. RIAO97 focuses on new problems in information retrieval, filtering, and dissemination resulting from the recent profusion and extensions of networks. In particular, RIAO97 brings together search specialists and web-based media specialists to consider how searching can best be accomplished in the context of the proliferation of web sites, content formats, browsing modalities, amount of data accessible, and number of user accesses. ------------------------------------------ FINAL PROGRAM AND SCHEDULE GENERAL SESSION June 25, 1997 ------------------------------------------ 9:00-9:15 a.m WELCOMING STATEMENT Prof. P.R. Belanger Vice Principal, Research McGill University, Canada 9:15-9:30 a.m RIAO97' Introduction J. Thuiller Professor at the College de France President of the C.I.D. 9:30-10:00 a.m Invited Speaker GILS Projet N. Brodie National Library of Canada, Ottawa, Canada --------------------------------- SESSION 1: INFORMATION DISCOVERY --------------------------------- Chairman: L. Devroye, McGill University 10:00-10:25 a.m "The Do-I-Care Agent: Effective Social Discovery and Filtering on the Web" M.S. Ackerman, B. Starr, M. Pazzani University of California, USA 10:25-10:50 a.m "Mining Information In Order To Extract Hidden And Strategical Information" T.Dkaki, B. Dousset, J. Mothe Universite P. Sabatier, Toulouse, France 10:50-11:20 a.m Break and Demonstrations ---------------------------------------------------------- SESSION 2: VISUALISATION TOOLS FOR INFORMATION NAVIGATION ---------------------------------------------------------- Chairman: S. Tohme, ENST France 11:20-11:45 a.m "Design Issues for World Wide Web Navigation Visualisation Tools" A. Cockburn, S. Jones University of Canterbury, University of Waikato, New Zealand 11:45-12:10 a.m "Footprints: History-Rich Web Browsing" A. Wexelblat, P. Maes Media Lab, Massachusetts Institute of Technology, USA 12:10-12:35 p.m "Using terminological base for Term-based information retrieval" J-Y Nie, Universite de Montreal, Canada 12:35-2:05 p.m LUNCH ------------------------------------------------ SESSION 3: AUTOMATIC ABSTRACTING, REPOSITORIES ------------------------------------------------ Chairman: G. Grefenstette, Rank Xerox Research Centre 2:05-2:30 p.m "Development of a Document Summarization System for Effective Information Services" D. H. Jang, S. H. Myaeng Chungnam National University, Taejon, Korea 2:30-2:55 p.m "Automatic summarization on the Web? RAFI: A system for summarizing using indicating fragments" A. Lehmam, Universite de Nancy II, France 2:55-3:20 p.m "Towards Sophisticated Wrapping of Web-based Information Repositories" B. Chidlovskii, U. M. Borghoff, P.Y. Chevalier Rank Xerox Research Centre, France 3:20-3:45 p.m "Annotating the World Wide Web using Natural Language" B. Katz, Massachussetts Institute of Technology, USA 3:45-4:15 p.m Break and Demonstrations --------------------------------------------------------- SESSION 4: LINGUISTIC APPROACH FOR INFORMATION RETRIEVAL --------------------------------------------------------- Chairman: R. Cencioni, European Community 4:15-4:40 p.m "IRENA: Information Retrieval Engine based on Natural Language Analysis" A.T. Arampatzis ,T. Tsoris, C.H.A. Koster. Patras, Greece and Netherlands 4:40-5:05 p.m "The Effect of Syntactic Phrase Indexing on Retrieval Performance for Dutch Texts" R. Pohlmann , W. Kraaij Utrecht University, TNO-TPD, Netherlands 5:05-5:30 p.m "Using Simulated Annealing to Understand Natural Language Texts" S.A. Laribi, G. Desrocques, A. Laribi, J.C. Bassano Universite d'Orleans, France. Geneva Univeristy, Switzerland 5:30-5:55 p.m "An Analysis of Statistical and Syntactic Phrases" M. Mitra, C. Buckley, A. Singhal, C. Cardie Cornell University, USA ----------------------------------- June 26, 1997 SESSION 5: MULTILINGUAL APPROACH ---------------------------------- Chairman: C. Fluhr, Commissariat a l'Energie Atomique. France 9:00-9:25 a.m "Multi-Language Text Indexing for Internet Retrieval" M.Wechsler, P. Sheridan, P. Schauble Swiss Federal Institute of Technology, Zurich, Switzerland 9:25-9:50 a.m "Adaptative Filtering of Multilingual Document Streams" D.W. Oard, University of Maryland, USA 9:50-10:15 a.m "A domain Specific Lexicon Acquisition Tool for Cross-Language Information Retrieval" D. Hiemstra, F. de Jong, W. Kraaij CTIT, Twente University, Netherlands 10:15-10:45 a.m Break and Demonstrations ---------------------------- SESSION 6: IRS ARCHITECTURE ---------------------------- Chairman: R. Marcus, Massachusetts Institute of Technology (USA) 10:45-11:10 a.m "EAGLE: An Extensible Architecture for General Linguistic Engineering" B. Baldwin , C. Doran, J.C. Reynar 11:10-11:35 a.m "Information Retrieval On The Word Wide Web using a Decision Making System" F. Corvaisier, A. Mille, J.M. Pinon, INSA, Lyon, France 11:35-12:00 a.m "A Multiagent Architecture for Information Retrieval on the World-Wide Web" V.N. Gudivada, S.P. Tolety Wayne State University, University of Missouri. USA 12:00-1:30 p.m LUNCH --------------------------------- SESSION 7: INFORMATION EXTRACTION --------------------------------- Chairman: C. Jacquemin, Universite de Nantes. France 1:30-2:45 p.m Panel TREC in Many Languages * TREC-5: English D. Harman National Institute of Standards and Technology, USA * TREC-5: Spanish and Chinese A. Smeaton Dublin City University, Ireland *The Amaryllis Project C. Fluhr Commissariat a l'Energie Atomique (CEA), France 2:45-3:10 p.m "Integrated text categorisation and information extraction using pattern matching and linguistic processing" W. J. Black, L. Gilardoni, F. Rinaldi, R Dressel UMIST, UK. Quinary Spa, Italy 3:10-3:35 p.m "On-Line Resource Discovery using Natural Language" O.R. Zaine , A. Fall, S. Rochefort, V. Dahl, P. Tarau Simon Fraser University, Canada 3:35-4:00 p.m "Coupling information retrieval and information extraction: A new text technology for gathering information from the web" R. Gaizauskas, A.M. Robertson University of Sheffield, UK 4:00-4:30 p.m Break and Demonstrations ---------------------------------------- SESSION 8: DOCUMENT / RELEVANCE RANKING ---------------------------------------- Chairman: D.J. Harman, National Institute of Standards and Technology, USA. 4:30-4:55 p.m "Summarizing Similarities and differences Among Related Documents" I. Mani , E. Bloedorn The MITRE Corporation, USA 4:55-5:20 p.m "Relevance Ranking for One to Three Term Queries" C.L.A. Clarke , G.V. Cormack, E.A. Tudhope University of Toronto. Canada 5:20-5:45 p.m "Space Optimizations for Total Ranking" D.R. Cutting, J.O. Pedersen Excite Inc., USA 5:45-6:10 p.m "A Similarity-Based Agent for Internet Searching" T.G. Rose, P. J. Wyard, Canon, UK ------------------------------- June 27, 1997 SESSION 9: IRS ARCHITECTURE- II ------------------------------- Chairman: U. Heid, Stuttgart University, Germany 9:00-9:25 a.m "ARACHNID: Adaptive Retrieval Agents Choosing Heuristic Neighborhoods for Information Discovery" F. Menczer , R.K. Belew University of California, USA 9:25-9:50 a.m "Cobra: A new approach to IR System design" T. Mills, K. Moody, K. Rodden University of Cambridge Computer Laboratory. UK 9:50-10:15 a.m "Interactive Image Retrieval by Means of Abductive Inference" A. Muller, A. Everts GMD. Germany 10:15-10:45 a.m Break and Demonstrations --------------------------------- SESSION 10: QUERY REFORMULATION --------------------------------- Chairman: J.C. Bassano, Universite d'Orleans France. 10:45-11:10 a.m "Query modification based on relevance backpropagation" M. Boughanem, C. Soule-Dupuy MSI, Universite de Limoges, France 11:10-11:35 a.m "Query ReFormulation on the Internet: Empirical Data and the Hyperindex Search Engine" P.D. Bruza , S. Dennis Queensland University of Technology, Australia 11:35-12:00 a.m SQLET: Short Query Linguistic Expansion Techniques: Palliating One or Two-word Queries by Providing Intermediate Structure to WWW Pages G. Grefenstette Rank Xerox Research Centre, Grenoble, France 12:00-1:30 p.m LUNCH --------------------------------- SESSION 11: INFORMATION FILTERING --------------------------------- Chairman: V. Semenova, ANALIT, Moscow 1:30-1:55 p.m "Probabilistic Learning for Information Filtering" G. Amati , F. Crestani, F. Ubaldini, S. de Nardis Fondazione Ugo Bordoni, Universita di Padova, Universita di Roma, Italy 1:55-2:20 p.m "Using Syntactic Information in Document Filtering: A Comparative Study of Part-of-speech Tagging and Supertagging" R. Chandrasekar , B. Srinivas University of Pennsylvania, USA 2:20-2:45 p.m "Querying Hierarchical Text and Acyclic Hypertext with Generalized Context-Free Grammars" Y. Marcoux, M. Sevigny Universite de Montreal, Canada 2:45-3:15 p.m Break and Demonstrations ------------------------------------- SESSION 12: INFORMATION EXTRACTION II ------------------------------------- Chairman: J.P. Haton, Universite Henri-Poincare, Nancy I, France 3:15-3:40 p.m "A probabilistic model of Passage Categorization" M. Iwayama, T. Tokunaga Advanced Research Laboratory, Japan 3:40-4:05 p.m "Knowledge Discovery From Natural Language Texts" U. Hahn, K. Schnattinger Freiburg University, Germany 4:05-4:30 p.m "Extraction of Index Words from Manuals" H. Nakagawa Yokohama National University, Japan -------------------------------- SESSION 13: JUDICIAL PROBLEMS -------------------------------- Chairman: J. Perriere, C.I.D. France 4:30-4:55 p.m "Limits of Using Data" Maitre Feral-Schuhl Ordre Des Avocats du Barreau de Paris, France. 4:55-5:20 p.m "Data Security" Maitre G. Arendt President of Computer Commission of the International Union of Lawyers, Luxemburg 5:20-5:45 p.m "Security Concerns On The Intranet" Maitre Landry Montreal, Canada. Union Internationale des Avocats. 5:45-6:15 p.m Conclusions - --------- For more information, please see our web page: http://www.sciences.univ-nantes.fr/RIAO97/ - ---------Mail to author|Respond to list|Read more issues|LINGUIST home page|Top of issue