LINGUIST List 8.838

Sat Jun 7 1997

Confs: History of ling, MUC-7

Editor for this issue: Ann Dizdar <annlinguistlist.org>

Directory

1. Lorenza Mondada, summer school on "History of linguistics"
2. marsh, MUC-7 CALL FOR PARTICIPATION

- ----

The Swiss Linguistics Society organizes a summer school on the topic
"Historical epistemology of linguistics", from September 1st to
September 5th, in Sion (Switzerland). This summer school consists in
two parts: 3 days of work-shops and 2 days of conference. For more
informations please contact Prof. Alain Berrendonner
(alain.berrendonnerunifr.ch). You will find here the french
presentation of the school and its program.

ECOLE D'ETE "EPISTEMOLOGIE HISTORIQUE DE LA LINGUISTIQUE" (Sion, 1-5
septembre 1997) Nous avons le plaisir de vous annoncer que la 3e Ecole
d'ete de la Societe Suisse de Linguistique (SSL), organisee en
collaboration avec l'Institut Kurt Bosch, se tiendra du 1er au 5
septembre 1997 Bramois pres de Sion (VS). Buts.

Le theme retenu, Epistemologie historique de la linguistique, tient
d'une part au fait que la SSL fete cette annee ses 50 ans
d'existence. Il nous a semble particuliarement opportun, l'occasion de
ce jubile, de jeter un regard retrospectif sur l'histoire de notre
discipline.

D'autre part, l'objectif des Ecoles d'ete de la SSL est d'apporter un
complement de formation aux etudiants, doctorands et jeunes
chercheurs, en diffusant desinformations qui relevent de domaines de
specialite peu ou pas representes dans les universites suisses. Or,
l'enseignement de l'histoire et de l'epistemologie de la linguistique
est loin d'tre assur partout de faon systematique. Il nous est donc
apparu qu'en choisissant ce theme, nous contribuions combler utilement
une lacune institutionnelle. Le but de l'Ecole est non seulement de
fournir aux participants des connaissances sur le passe de la
linguistique (theories, doctrines, methodes...), mais aussi de
favoriser l'exchange d'experiences entre chercheurs, la reflexion sur
les modes de construction des objets de science, et la mise en
perspective des pratiques cognitives.

Public vise.
L'ecole d'ete est destinee principalement aux doctorands et jeunes
chercheurs des universiters suisses, travaillant dans le domaine des
sciences du langage. Elle peut egalement interesser des specialistes
en epistemologie, histoire des sciences, philosophie des sciences,
histoire des idees...

Organisation.
La premiere partie du programme comprend cinq demi-journees,
consacrees chacune un cours, suivi de discussion et/ou de travaux
diriges. La seconde partie prendra la forme d'un colloque, au cours
duquel divers sp=E9cialistes pr=E9senteront leurs resultats les plus
recents, et illustreront leur methodologie sur des faits precis. La
participation la totalite du programme est de rigueur.

Programme
Lundi 1er septembre.
Matin: Konrad KOERNER (Toronto): Sprachwissenschaftsgeschichte: Sinn,
Zweck und Methode.
Apres-midi: Claudine NORMAND (Paris X): Le positivisme en
linguistique.

Mardi 2 septembre.
Matin: D. BARATIN / F. DESBORDES: La pensee medievale et
antique. 
Apres-midi: J.-C. CHEVALIER: Grammaire scolaire vs grammaire gensra le
du XVIIe au XIXe sicle.

Mercredi 3 septembre.
Matin: B. SCHLIEBEN-LANGE (Tubingen): Kontinuitten und Broche in
sprachwissenschaftlichen Diskursen.
Apres-midi: Jubile de la SSL. Regards sur 50 ans de linguistique en
Suisse Conference de S. BOUQUET: L'histoire editoriale des textes de
linguistique generale de F. de Saussure.

Jeudi 4 septembre / Vendredi 5 septembre: Colloque
9h-9h45 D. SKILIAN (Zagreb): La pensee linguistique grecque avant
Socrate.
9h45-10h30	R.H. ROBINS (Londres): Les grammairiens byzantins.
10h30-11h	pause
11h-11h45 R. AMACKER (Geneve): Sur les formes de l'argumentation
chez les grammairiens latins.
12h		repas
14h-14h45 O. POMBO (Lisbonne): La th=E9orie Leibnizienne de la pensee
aveugle en tant que perspective sur quelques apories linguistiques de
la modernite.
14h45-15h30 J. BREUILLARD (Lyon III): Etre linguiste en Russie au
XVIIIe siecle: la decouverte de la diversite des langues.
15h30-16h	pause
16h-16h45 J. TRABANT (Berlin): Mithridates: de Conrad Gesner
Adelung/Vater.
16h45-17h30 L. FORMIGARI (Rome): Epistemologie de la linguistique chez
Steinthal. Vendredi 5 septembre.
9h-9h45	P. SWIGGERS (Louvain): Gillieron: structuraliste ou
anti-structuraliste?
9h45-10h30 A. FRYBA (Berne): Philologie und Sprachwissenschaft in der
Schweiz am Anfang des 20. Jahrhunderts.
10h30-11h	pause
11h-11h45 J. FEHR (Zurich): Semiologie im Spannungsfeld von Sprache
und Schrift.
12h		repas
14h-14h45 D. GAMBARARA (Univ. di Calabria): L'ouverture historique de
l'espace de la linguistique.
14h45-15h30 G. BERGOUGNIOUX (Orl=E9ans): La langue et le cerveau:
genese de l'aphasie.
15h30-16h	pause
16h-16h45 K. WIPPICH-HORACKOVA (Prague): Sprachlehrwerke als Spiegel
der Sprachwissenschaft.

Inscription.
Le nombre de participants est limite a 30.
Les frais d'inscription, couvrant l'hebergement et les repas, se
montent a Sfr. 500.-
Grce a une subvention de l'Academie Suisse des Sciences humaines, les
frais d'inscription d'une dizaine de participants etudiants peuvent
tre pris en charge par le budget de la SSL. La demande doit en otre
faite au moment de l'inscription.

Pour s'inscrire, contacter au plus vite:
Societe suisse de linguistique
(Ecole d'ete)
Prof. A. Berrendonner
Criblet 13,
1700 Fribourg
e-mail: alain.berrendonnerunifr.ch

 * * * CALL FOR PARTICIPATION * * *

 SEVENTH MESSAGE UNDERSTANDING SYSTEM EVALUATION
 AND MESSAGE UNDERSTANDING CONFERENCE (MUC-7)

 Evaluation: 2-6 March 1998
 Conference: April 1998
 Washington, D.C. area


 Sponsored by:

 The Human Language Systems Tipster Text Program of the
 Defense Advanced Research Projects Agency
 Information Technology Office
 (DARPA/ITO)


 The Message Understanding Conferences have provided on ongoing
forum for assessing the state of the art and practice in text analysis
technology and for exchanging information on innovative computational
techniques in the context of fully implemented systems that perform
realistic tasks. The evaluations have provided researchers and
potential sponsors and customers with a quantitative means to
appreciate the strengths and weaknesses of the technologies, and the
results reported on at the conferences have sparked customer interest
in the potential utility of the technologies.

 The Seventh Message Understanding Conference (MUC-7) will provide
an opportunity for both new and experienced MUC participants to
participate in a flexible evaluation, suited to development needs and
abilities. It will provide:

* Opportunity to select among a variety of tasks: Named Entity
 (NE), Coreference (CO), Template Element (TE), Template
 Relationship (TR) and Scenario Template (ST).
* Two tasks for evaluating component technologies (NE and
 CO), which use Standard Generalized Markup Language (SGML)
 as output format
* Redesigned Information Extraction (IE) task, with two
 domain-independent subtasks (TE and TR) separated from
 domain-dependent subtask (ST).
* Emphases of ST task on portability and on minimizing
 human resources required to participate in the evaluation.
* Three experimental tracks to explore new data sets and tasks.

 Participation in MUC-7 is actively sought from both new and
veteran organizations. With the new and redesigned evaluation tasks,
MUC-7 offers a good opportunity for organizations to try out new ideas
for handling NLP problems that are of both scientific and practical
interest without having to participate in the entire range of tasks.

 The conference itself will consist primarily of presentations and
discussions of innovative techniques, system design, and test results.
There will also be an opportunity for participants to demo their
evaluation systems. Attendance at the conference is limited to
evaluation participants and to guests invited by the DARPA Tipster
Text Program. A conference proceedings, including test results, will
be published.


SCHEDULE:

	1 July 97: Application deadline for participation
	 
	15 July 97: Release of NE, CO, TE, TR, and example ST training
		 data and scorer

	8 September 97: Release of Dry Run ST task definition,
			training data, and scorer

	29 Sept - 3 Oct 97: MUC-7 Dry Run (all participants)

	6 February 98: Release of formal test ST task definition,
		 training data, and scorer

	2-6 March 98: MUC-7 Formal Run

	7-9 April 98: 7th Message Understanding Conference (tentative
 dates)


DATA AND TASK DESCRIPTION:

 The texts to be used for system development and testing are news
service articles from the New York Times News Service, supplied by the
Linguistic Data Consortium (LDC) [ldcldc.upenn.edu]. Training, dry
run, and test data for all the tasks are extracted from a corpus of
approximately 158,000 articles. Sets of articles to be used in the
MUC-7 evaluation will be distributed via ftp upon payment of a one
time fee of $100 and upon signing of a user agreement for the use of
these texts. The user agreement can be retrieved from the LDC catalog
(Evaluation Agreements). The URL for the LDC home page is:
http://www.ldc.upenn.edu.

 Five separate evaluations will be conducted as part of MUC-7.
The definition of these evaluations has been worked out since late
1996 by members of the MUC-7 Planning Committee. The evaluations may
be viewed as capturing the results of text analysis at various levels
of aggregation of information:
 
* Named Entity (NE) requires only that the system under
 evaluation identify each bit of pertinent information in
 isolation from all others.
* Coreference (CO) requires connecting all references to
 "identical" entities.
* Template Element (TE) requires grouping entity attributes
 together into entity "objects."
* Template Relationship (TR) requires identifying relationships
 between template elements.
* Scenario Template (ST) requires identifying instances of a
 task-specific event and identifying event attributes,
 including entities that fill some role in the event; the
 overall information content is captured via interlinked
 "objects."
* Experimental tracks using new data sets are variants
 of the NE task. The task definition is the same as for the 
 basic NE task, but the texts are different.
* Experimental track involving a new task is a simplified 
 version of the TE task.

 Key things to note about each evaluation task:
 
* NE covers named organizations, people, and locations, along
 with date/time expressions and monetary and percentage
 expressions; it requires production of SGML tags as output.
* CO covers noun phrases (common and proper) and personal
 pronouns that are "identical" in their reference; it requires
 production of SGML tags as output; the tags for coreferring
 strings form "equivalance" classes, which are used for
 scoring.
* TE covers organizations, persons, and artifacts, which are
 captured in the form of template "objects" consisting of a
 predefined set of attributes.
* TR covers relationships among template elements, including
 location and time relationships, which are captured in the
 form of template "relations" consisting of a relationship
 and the template elements participating in that
 relationship. TR is a new task for MUC-7.
* ST covers a particular scenario, which is kept secret until
 one month prior to testing in order to focus on system
 portability; however, the generalized structure of a
 scenario template is predefined, and example scenarios are
 available for participants to examine. This task is domain
 dependent.
* Tasks for the experimental tracks are derived from NE and TE.

 There is a World Wide Web site that allows automated testing
following the rules of MUC-6. It will be of particular value to new
participants. The website is password protected and you need to be
licensed to access the ACL/DCI disk from the LDC to obtain a password
from chinchorgso.saic.com. MUC-6 articles were taken from the ACL/DCI
disk.

 An anonymous ftp site will be available for downloading MUC-7
related material. This CFP and the MUC-7 Participant Agreement are
available to the public from the ftp site. Each participant (after
signing the LDC User Agreement and a MUC-7 participation agreement)
will receive a password to download the MUC-7 data, definitions, and
scoring software at the release times noted above.

 The URL of the website is http://muc.saic.com. The ftp site is
ftp.muc.saic.com. 


TEST PROTOCOL AND EVALUATION CRITERIA:

 MUC-7 participants may elect to do one or any combination of
tasks and experimental tracks. Participants will have access to
shared resources such as the training texts and annotations/templates,
task documentation, and scoring software.

 All MUC-7 participants are encouraged to participate in the dry
run and take advantage of material available.

 The formal test will be conducted during the first week in March.
It will be carried out by the participants at their own sites in
accordance with a prepared test procedure and the results submitted to
the ftp site for official scoring with the software prepared by SAIC
for MUC-7.

 Test sets used for the evaluations will consist of 100 texts,
with subsets for some of the tasks. There will be different data sets
for the dry run and the formal test.

 Systems will be evaluated using recall and precision metrics (all
tasks), F-measure (all tasks), and error-based metrics (all tasks
except CO). The computation of these metrics is based on the scoring
categories of correct, partial, incorrect, spurious, missing, and
noncommittal. MUC-7 participants will be able to familiarize
themselves with the evaluation criteria through usage of the
evaluation software, which will be released along with the training
data.


INSTRUCTIONS FOR RESPONDING TO THE CALL FOR PARTICIPATION:

 Organizations within and outside the U.S. are invited to respond
to this call for participation. By the time of the actual testing
phase of the evaluation, systems must be able to accept texts without
manual preprocessing, process them without human intervention, and
output annotations (NE, CO) or templates (TE, TR, ST) in the expected
format.

 Organizations should plan on allocating approximately two
person-months of effort for participation in the evaluation and
conference. It is understood that organizations will vary with
respect to experience with SGML text annotation, information
extraction, domain expertise/engineering, resources, contractual
demands/expectations, etc. Recognition of such factors will be made
in any analyses of the results.

 Organizations wishing to participate in the evaluation and
conference must respond by July 1, 1997 by submitting a short
statement of interest via email and a signed copy of the MUC-7
participation agreement via surface mail.

 1. The statement of interest should be submitted via email
to marshaic.nrl.navy.mil and should include the following:

 a. Evaluation task(s) (choose one or more)
 * Named Entity
 * Coreference
 * Template Element
 * Template Relationship
 * Scenario Template
 b. Primary point of contact. Please include name, surface
 and email addresses, and phone and fax numbers.
	 c. Does your site have a copy of the MUC-6 proceedings?

 2. The participation agreement can be downloaded from the
anonymous ftp site (ftp.muc.saic.com). A signed copy should be sent by
surface mail to Elaine Marsh, NRL - Code 5512, 4555 Overlook Ave. SW,
Washington, D.C. 20375-5337, USA.

If some questions cannot be deferred until the deadline for
responding to this call for participation has passed, you may send
them by email to Elaine Marsh (marshaic.nrl.navy.mil), WITH COPIES TO
Ralph Grishman (grishmancs.nyu.edu) and Nancy Chinchor
(chinchorgso.saic.com) to ensure that your message receives a timely
response from one of us.


MUC-7 PLANNING COMMITTEE:

 Ralph Grishman, New York University, program co-chair
 Elaine Marsh, Naval Research Laboratory, program co-chair
 Chinatsu Aone, Systems Research and Applications
 Lois Childs, Lockheed Martin
 Nancy Chinchor, Science Applications International
 Jim Cowie, New Mexico State University
 Rob Gaizauskas, University of Sheffield
 Megumi Kameyama, SRI International
 Tom Keenan, U.S. Department of Defense
 Boyan Onyshkevych, U.S. Department of Defense
 Martha Palmer, University of Pennsylvania
 Beth Sundheim, NCCOSC NRaD
 Marc Vilain, MITRE
 Ralph Weischedel, BBN Systems and Technologies