You are here: > ESRI Forums > arcgis desktop discussion forums > Thread Replies

ArcGIS Desktop Discussion Forums

ArcGIS Desktop - Extension - Network Analyst forum

Build Network Dataset Failed   Pratik Sanyal Oct 28, 2007
Re: Build Network Dataset Failed   Alan Hatakeyama Oct 29, 2007
Re: Build Network Dataset Failed   Pratik Sanyal Oct 29, 2007
Re: Build Network Dataset Failed   Alan Hatakeyama Oct 29, 2007
Re: Build Network Dataset Failed   Pratik Sanyal Oct 29, 2007
Re: Build Network Dataset Failed   Alan Hatakeyama Oct 30, 2007
Re: Build Network Dataset Failed   Pratik Sanyal Oct 30, 2007
Re: Build Network Dataset Failed   Alan Hatakeyama Oct 30, 2007
Re: Build Network Dataset Failed   Pratik Sanyal Oct 30, 2007
Re: Build Network Dataset Failed   Pratik Sanyal Nov 08, 2007
Re: Build Network Dataset Failed   Alan Hatakeyama Nov 08, 2007
Re: Build Network Dataset Failed   Pratik Sanyal Nov 09, 2007
Re: Build Network Dataset Failed   Pratik Sanyal Nov 09, 2007
Re: Build Network Dataset Failed   Alan Hatakeyama Nov 09, 2007
Re: Build Network Dataset Failed   Pratik Sanyal Nov 09, 2007
Re: Build Network Dataset Failed   Alan Hatakeyama Nov 09, 2007
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Build Network Dataset Failed 
Author Pratik Sanyal 
Date Oct 28, 2007 
Message When I tried build a network dataset from Streets layer with 11586078 features and LogicalNetworkCacheSize as 8000 it worked. Now I am trying to build a network dataset of whole of US streets (38968240 features). I have set LogicalNetworkCacheSize in registry to 600000000. When attempted the build it failed saying the user set logical network memory quota has been reached. Could you please explain how this network memory is linked to LogicalNetworkCacheSize. What could be the reason it failed in my case ?

Thanks,
Pratik 
  Attributes.txt (opens in new window)
Build.txt (opens in new window)
 
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Alan Hatakeyama 
Date Oct 29, 2007 
Message Hi Pratik,

According to http://support.esri.com/index.cfm?fa=knowledgebase.techarticles.articleShow&d=30563 the LogicalNetworkCache size is the size in MB. Did you mean to set the cache to 600 MB (instead of 600 million MB), or is there a reason why you need 600 million MB (or 600 TB) of cache?

Just FYI, I've successfully built a US network dataset with a cache size of 4096 MB (4 GB).


Alan 
   
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Pratik Sanyal 
Date Oct 29, 2007 
Message Hi Alan,

Thanks for your reply. I had set the LogicalNetworkCacheSize to 8000, 16000, 40000, 60000000 MB and tried several times still I face the issue. It looks like the value is not getting reflected (Attributes.txt). I am try to build it from a street layer inside a File Geodatabase. I would like to know on what geodatabase you had build the whole of US street network and number of features in the refernced street layer ? Also, by cache to 4096 MB I guess you mean LogicalNetworkCacheSize in registry. Is that correct ? My system has a RAM size of 2 GB.

I appreciate your help.

Thanks,
Pratik
Cell: 201.673.7870 
   
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Alan Hatakeyama 
Date Oct 29, 2007 
Message Q: I would like to know on what geodatabase you had build the whole of US street network and number of features in the refernced street layer ?
A: My data was more than 41 million line features in a file geodatabase.

Q: Also, by cache to 4096 MB I guess you mean LogicalNetworkCacheSize in registry. Is that correct ?
A: Yes. 
   
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Pratik Sanyal 
Date Oct 29, 2007 
Message Thanks a lot for your inputs.

A couple of follow up queries,

Q1. What is the RAM size of the machine from which you had build the whole US street network ?

Q2. I am using ArcCatalog to perform the Network Dataset build. Did you use the same ?

Q3. For 39 million features, the cache 4096 MB should do I guess. Did you get to figure the link betweeen LogicalNetworkCacheSize and the memory MB logged in the Attributes.txt

Thanks !!
Pratik 
   
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Alan Hatakeyama 
Date Oct 30, 2007 
Message A1: 2 GB.

A2: Yes.

A3: I'm not exactly sure, but I know that as the network build progresses, it can need more memory to handle the large number of elements in the network, or large number of network attributes.


Alan 
   
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Pratik Sanyal 
Date Oct 30, 2007 
Message Thanks a lot for your response to my queries.

Could you please check the memory size in MB for in the Attributes.txt file for the build you made ?

For example for me it was,

13 States build (which was sucessful):
Updating Attributes: 0.0 seconds (0.0 total) - 0 MB memory [1:17:42 AM]

Source ALL_STREETS [1 ... 11586078]: 392.5 seconds (392.5 total) - 4294967295 MB memory
Cached feature ID range [1 .. 1000000]: 54.3 seconds (446.8 total) - 4294967295 MB memory
Feature cursor search: 2.2 seconds (449.0 total) - 4294967295 MB memory

50 States build (which failed):
Updating Attributes: 0.0 seconds (0.0 total) - 0 MB memory [8:24:06 PM]

Source TRANS_STREET_GDT [1 ... 38968240]: 867.4 seconds (867.4 total) - 0 MB memory
Cached feature ID range [1 .. 1000000]: 67.6 seconds (935.0 total) - 0 MB memory
Feature cursor search: 2.1 seconds (937.2 total) - 0 MB memory

I am planning to attempt the build again with LogicalCacheNetworkSize set to 4096 MB in decimal value.

Pratik 
   
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Alan Hatakeyama 
Date Oct 30, 2007 
Message Unfortuantely, it's been a while since I've built my US network dataset, so I no longer have the log file from when I did it.

"4294967295 MB memory" doesn't seem right, as that's a LOT of memory! I noticed that the number 4294967295 is one less than 2^32, so that number is probably in error.


Alan 
   
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Pratik Sanyal 
Date Oct 30, 2007 
Message Thanks Alan.

Yeah, that number does not look realistic though.
I am planning start a build tonight with 4096 MB.

Thanks,
Pratik 
   
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Pratik Sanyal 
Date Nov 08, 2007 
Message Hi Alan,

How are you doing ?

I was able to overcome the logical cache memory error by setting it to 8000 mb. I was wrongly setting the value under HKEY_LOCAL_MACHINE before. Now I do not get the error. However, I encountered "Out Of Memory" runtime error when it tries to build the Attributes.txt. I have attached my Attributes.txt.

I have a 2GB RAM machine and Virtual memory set to 3092 MB. Looks like the attributes build trying allocated more than 2 GB memory and gets the run time error. I know were successful to build a dataset with 41 million features in 2GB RAM machine. I would like to know,

Q1. How many attributes did you have in the network dataset build and what are they ?

Q2. How much virtual memory did you use during the build ?

Thanks a lot for your help !!

Thanks,
Pratik

 
  Attributes.txt (opens in new window)
 
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Alan Hatakeyama 
Date Nov 08, 2007 
Message Hi Pratik,

Good to hear that you got the Logical Network cache size resized with the proper registry key.

My network dataset has eight attributes: Time cost, Distance cost, Hierarchy, One-Way street restriction, Toll Road restriction, Ferry restriction, Turn restriction, and Road Class descriptor. All of these attributes used the field evaluator, either as a single field or in an expression.

My machine started with 2 GB of virtual memory and had a max of 4 GB. I don't remember what the max usage was when I built it.


Alan 
   
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Pratik Sanyal 
Date Nov 09, 2007 
Message Hi Alan,

Thanks for your inputs !!

I am planning to run a build with 3 GB initial virtual memory and max to 4 Gb. I have five network attributes: Time, Length, Hierarchy, Road Class, One-way.

My system is running on Windows 2000 Professional. I would like to know the OS in the system you had used to build the network dataset.

Also, do you have an idea how the network Attributes estimate the memory.

For, 11 million features - 5 attributes - used max 590 Mb memory

39 million features - 5 attributes - used upto 1094 Mb and then got Out of memory

39 million features - 1 attribte (Length) - used upto 507 Mb

I am trying to understand, how the memory allocation / estimation is happening.

Any inputs on this ?

Thanks,
Pratik 
  Attributes_1 attribute.txt (opens in new window)
 
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Pratik Sanyal 
Date Nov 09, 2007 
Message Name: US_STREETS_ND
Type: Geodatabase-Based Network Dataset

Sources:
Edge Sources:
TRANS_STREET_GDT

Connectivity:
Group 1:
Edge Connectivity:
TRANS_STREET_GDT (End Point)

Elevation Fields:
Edge Elevation Fields: (From End, To End):
TRANS_STREET_GDT: (F_ZLEV, T_ZLEV)

Turns:


Attributes:
Oneway:
Usage Type: Restriction
Data Type: Boolean
Units Type: Unknown
Use by Default: True
Source Attribute Evaluators:
TRANS_STREET_GDT (From-To): Field -
Prelogic:
restricted = False
Select Case UCase([One_way])
Case "N", "TF", "T": restricted = True
End Select
Expression: restricted
TRANS_STREET_GDT (To-From): Field -
Prelogic:
restricted = False
Select Case UCase([One_way])
Case "N", "FT", "F": restricted = True
End Select
Expression: restricted
Default Attribute Evaluators:
Default Edges: Constant - Traversable
Default Junctions: Constant - Traversable
Default Turns: Constant - Traversable
Time:
Usage Type: Cost
Data Type: Double
Units Type: Minutes
Use by Default: True
Source Attribute Evaluators:
TRANS_STREET_GDT (From-To): Field - [FT_Cost]
TRANS_STREET_GDT (To-From): Field - [TF_Cost]
Default Attribute Evaluators:
Default Edges: Constant - 0
Default Junctions: Constant - 0
Default Turns: Constant - 0
RoadClass:
Usage Type: Descriptor
Data Type: Integer
Units Type: Unknown
Use by Default: False
Source Attribute Evaluators:
TRANS_STREET_GDT (From-To): Field -
Prelogic:
rc = 1 'Local road
Select Case UCase([CFCC])
Case "A10", "A11", "A12", "A13", "A14", "A15", "A16", "A17", "A18", "A19"
rc = 2 'Highway
Case "A60", "A63"
rc = 3 'Ramp
Case "A65", "A66", "A68", "A69"
rc = 4 'Ferry
Case "A62"
rc = 5 'Roundabout
End Select
Expression: rc
TRANS_STREET_GDT (To-From): Field -
Prelogic:
rc = 1 'Local road
Select Case UCase([CFCC])
Case "A10", "A11", "A12", "A13", "A14", "A15", "A16", "A17", "A18", "A19"
rc = 2 'Highway
Case "A60", "A63"
rc = 3 'Ramp
Case "A65", "A66", "A68", "A69"
rc = 4 'Ferry
Case "A62"
rc = 5 'Roundabout
End Select
Expression: rc
Default Attribute Evaluators:
Default Edges: Constant - 0
Default Junctions: Constant - 0
Default Turns: Constant - 0
Hierarchy:
Usage Type: Hierarchy
Data Type: Integer
Units Type: Unknown
Use by Default: True
Hierarchy Ranges:
Primary Roads: up to 1
Secondary Roads: 2 - 2
Local Roads: 3 and higher
Source Attribute Evaluators:
TRANS_STREET_GDT (From-To): Field -
Prelogic:
res = 3

if [ACC] = "1" or [ACC] = "2" then
res = 1
elseif [ACC] = "3" then
res = 2
elseif [CFCC] < "A40" or [CFCC] = "A60" or [CFCC] = "A63" or [CFCC] = "A68" or [CFCC] = "A69" or [CFCC] = "A45" then
res = 2
end if
Expression: res
TRANS_STREET_GDT (To-From): Field -
Prelogic:
res = 3

if [ACC] = "1" or [ACC] = "2" then
res = 1
elseif [ACC] = "3" then
res = 2
elseif [CFCC] < "A40" or [CFCC] = "A60" or [CFCC] = "A63" or [CFCC] = "A68" or [CFCC] = "A69" or [CFCC] = "A45" then
res = 2
end if
Expression: res
Default Attribute Evaluators:
Default Edges: Constant - 0
Default Junctions: Constant - 0
Default Turns: Constant - 0
Length:
Usage Type: Cost
Data Type: Double
Units Type: Miles
Use by Default: False
Source Attribute Evaluators:
TRANS_STREET_GDT (From-To): Field - [SHAPE_Length]
TRANS_STREET_GDT (To-From): Field - [SHAPE_Length]
Default Attribute Evaluators:
Default Edges: Constant - 0
Default Junctions: Constant - 0
Default Turns: Constant - 0

Directions:
Directions Ready: Yes
General Directions:
Display Length Units: Miles
Length Attribute: Length
Time Attribute: Time
Road Class Attribute: RoadClass
Source Directions:
TRANS_STREET_GDT:
Street Name Fields:
Primary:
Prefix: PREFIX
Name: NAME
Suffix Type: TYPE
Suffix: SUFFIX
Shields:
Shield Field Type: Single Field
Shield Description Field: SHIELD

-------- This is the summary of my network dataset.

Thanks,
Pratik 
   
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Alan Hatakeyama 
Date Nov 09, 2007 
Message Hi Pratik,

I used Windows XP.

My best guess would be to take the 11 million case and "scale" it up to the full case. So you would probably need 590 MB 39 / 11 = ~2100 MB of memory. To be safe, you can probably do the same estimate by scaling up your 1 attribute case, which would result in 2535 MB (though it may not be as accurate as not all of the attributes are of the same data type). Either way, you shouldn't cut it close, and leave some wiggle room.

Keep in mind that the memory usage numbers (590 MB, 507 MB, ~2100 MB, etc.) reflect what is being used ONLY by network building, and not the total memory consumption of your computer -- this figure excludes any memory consumption currently used or will be used by your OS and other applications that you or your system may be running (ArcCatalog's usage is also excluded from this figure).


Alan 
   
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Pratik Sanyal 
Date Nov 09, 2007 
Message Hi Alan,

Thanks for your inputs !!

I have increased the virtual memory of my system and started the build with 5 attributes.

I am also planning to request my manager to increase the RAM if possible.

Do you think the increase the virtual memory size to 4 Gb would help ? Should I try changing the OS to XP ?


Thanks,
Pratik 
   
Report Inappropriate Content • Top • Print • This Forum is closed for replies.    
Subject Re: Build Network Dataset Failed 
Author Alan Hatakeyama 
Date Nov 09, 2007 
Message The more memory, the more likely it would succeed. Be sure that the hard drive(s) that you allocate the virtual memory on can afford the space.

I don't think that the OS matters too much.


Alan