El grupo al cual envías entradas es un grupo Usenet. Si envías mensajes a este grupo, cualquier usuario de Internet podrá ver tu dirección de correo electrónico
I have done a small benchmark with routing implementation with sf1.2 and sf1.3.
I have created 4 cache files with 500 routes in each files. - test 1 : symfony 1.2 , routing.yml.php - object creation in an array - test 2 : version "cache" symfony 1.2, serialized array with compiled routes from test 1 - test 3 : version swOptimizeRoutesPlugin : array of serialized routes - test 4 : version symfony 1.3, a big array
Here the result (I have been running this test with php5.3 / macport):
# TEST 1 : sf1.2 / sfPatternRouting - object creation # TEST 1 - file size: 398576.000000, include time: 0.022773, memory: 2544656 # TEST 2 : sf1.2 / sfPatternRouting - object deserialization as if cache is enabled # TEST 2 - file size: 398576.000000, include time: 0.032223, memory: 5787200 # TEST 3 : sf1.2 / swOptimizePatternRouting - array with serialized route informations # TEST 3 - file size: 426672.000000, include time: 0.004196, memory: 2183944 # TEST 4 : sf1.3 / sfPatternRouting like - full array # TEST 4 - file size: 652672.000000, include time: 0.036927, memory: *9573008*
My first conclusion is that sf1.3 take more memory than any other implementations without any speed gain. Of course I might missing something important ...
I was not able to get your code running easily.
Here a drop in zip file with my tests. I also noted you were not
always passing true to memory_get_usage. any reason for that?
To eliminate any chance for error i start with a routing.yml file
which i generate with 1000 routes (high number but not too
unrealistic)
After that I generate the 1.2 and 1.3 version of the confighandler file
Then i am loading this configuration into pattern routing and running
a query against them
I monkeypatched the sfPatternRouting to make them running isolated.
this should not have any impact on the measurements
1.2 uses less memory and is faster on generation. This is
understandable, because 1.3 moves the route compilation to here.
loading the config is really interesting. it does'nt take memory for
1.2 (i don't know why) and it is faster than in 1.3 (because there is
less to load, because less was generated). It takes TONS of memory for
1.3
then finding the route is WAY faster in 1.3 than in 1.2. However,
assuming this simple uri against simple route matching is ok for our
test, the gain in parse() does not offset the heavy lifting on loading
the configuration.
Also the memory consumption is not really nice in 1.3
@Fabien, any immediate ideas? We have to work on this one still
On Wed, Oct 28, 2009 at 4:46 PM, Thomas Rabaix <thomas.rab...@gmail.com> wrote:
> Hello,
> I have done a small benchmark with routing implementation with sf1.2
> and sf1.3.
> I have created 4 cache files with 500 routes in each files.
> - test 1 : symfony 1.2 , routing.yml.php - object creation in an
> array
> - test 2 : version "cache" symfony 1.2, serialized array with
> compiled routes from test 1
> - test 3 : version swOptimizeRoutesPlugin : array of serialized
> routes
> - test 4 : version symfony 1.3, a big array
> Here the result (I have been running this test with php5.3 / macport):
> # TEST 1 : sf1.2 / sfPatternRouting - object creation
> # TEST 1 - file size: 398576.000000, include time: 0.022773, memory:
> 2544656
> # TEST 2 : sf1.2 / sfPatternRouting - object deserialization as if
> cache is enabled
> # TEST 2 - file size: 398576.000000, include time: 0.032223, memory:
> 5787200
> # TEST 3 : sf1.2 / swOptimizePatternRouting - array with serialized
> route informations
> # TEST 3 - file size: 426672.000000, include time: 0.004196, memory:
> 2183944
> # TEST 4 : sf1.3 / sfPatternRouting like - full array
> # TEST 4 - file size: 652672.000000, include time: 0.036927, memory:
> *9573008*
> My first conclusion is that sf1.3 take more memory than any other
> implementations without any speed gain. Of course I might missing
> something important ...
Hi again,
I really dont know if this is a PHP glitch or something, but I have a
much better solution.
instead of generating the new sfRoute(...big array ...) stuff with the
config handler i do this:
So when this config file is generated all routes are compiled and
serialized to this file (this is similar to how 1.2 worked with
caching)
after doing so the file is just included in the pattern routing and
fills directly the $this->routes by unserializing.
I do not know why this is faster and consumes less memory but that is
it. I speculate that the hundreds of arrays the generated file
contains is just bad for php.
Here the results:
-> faster creation of config file & less memory. 1/3 execution time
and 1/5 memory consumption on loading. But I admit i do not trust the
memory figures.
With my change it is now half time for loading the config and about
the same memory (you have to add the memory in searching for 1.2
because it builds the routes then (lazy))
Also the generated routing.yml.php is about half of the current 1.3 size.
I attached the improved version. Please crosscheck
<fabian.la...@symfony-project.com> wrote:
> Hi Thomas,
> I was not able to get your code running easily.
> Here a drop in zip file with my tests. I also noted you were not
> always passing true to memory_get_usage. any reason for that?
> To eliminate any chance for error i start with a routing.yml file
> which i generate with 1000 routes (high number but not too
> unrealistic)
> After that I generate the 1.2 and 1.3 version of the confighandler file
> Then i am loading this configuration into pattern routing and running
> a query against them
> I monkeypatched the sfPatternRouting to make them running isolated.
> this should not have any impact on the measurements
> 1.2 uses less memory and is faster on generation. This is
> understandable, because 1.3 moves the route compilation to here.
> loading the config is really interesting. it does'nt take memory for
> 1.2 (i don't know why) and it is faster than in 1.3 (because there is
> less to load, because less was generated). It takes TONS of memory for
> 1.3
> then finding the route is WAY faster in 1.3 than in 1.2. However,
> assuming this simple uri against simple route matching is ok for our
> test, the gain in parse() does not offset the heavy lifting on loading
> the configuration.
> Also the memory consumption is not really nice in 1.3
> @Fabien, any immediate ideas? We have to work on this one still
> Fabian
> On Wed, Oct 28, 2009 at 4:46 PM, Thomas Rabaix <thomas.rab...@gmail.com> wrote:
>> Hello,
>> I have done a small benchmark with routing implementation with sf1.2
>> and sf1.3.
>> I have created 4 cache files with 500 routes in each files.
>> - test 1 : symfony 1.2 , routing.yml.php - object creation in an
>> array
>> - test 2 : version "cache" symfony 1.2, serialized array with
>> compiled routes from test 1
>> - test 3 : version swOptimizeRoutesPlugin : array of serialized
>> routes
>> - test 4 : version symfony 1.3, a big array
>> Here the result (I have been running this test with php5.3 / macport):
>> # TEST 1 : sf1.2 / sfPatternRouting - object creation
>> # TEST 1 - file size: 398576.000000, include time: 0.022773, memory:
>> 2544656
>> # TEST 2 : sf1.2 / sfPatternRouting - object deserialization as if
>> cache is enabled
>> # TEST 2 - file size: 398576.000000, include time: 0.032223, memory:
>> 5787200
>> # TEST 3 : sf1.2 / swOptimizePatternRouting - array with serialized
>> route informations
>> # TEST 3 - file size: 426672.000000, include time: 0.004196, memory:
>> 2183944
>> # TEST 4 : sf1.3 / sfPatternRouting like - full array
>> # TEST 4 - file size: 652672.000000, include time: 0.036927, memory:
>> *9573008*
>> My first conclusion is that sf1.3 take more memory than any other
>> implementations without any speed gain. Of course I might missing
>> something important ...
I am glad that you get the time to look at this issue. I don't know if you
have a look into the plugin, but I don't think symfony or either PHP can
deal with this work : loading this information into memory is too slow
: 304ms to load 1000 routes (maybe it will be 90ms for 200 routes).
This logic can be easily ported into the webserver's configuration (maybe
only in production) with mod_rewrite, so the matching is not done at the
symfony/php level.
The next thing, route information should be loaded on demand [1], each page
can have an average of 20-30 routes loaded (why the other routes should be
loaded ?). But there is another problem, PHP is slow to deserialize string,
so loading on demand routes will be slow...
Now, I am pretty confident that the native php serializer is not suitable
for routing, maybe the sfRoute should have its own serializer/deserializer.
So in this case loading routes can be quicker. I have not try this because
this will require to patch symfony, I don't want to propose this as a
plugin.
[1] on demand : the cache system only hold an array of serialized routes :
array(
'my_route_1' => array('sfDoctrineRoute', "serialized data"),
'my_route_2' => array('sfDoctrineRoute', "serialized data"),
[...]
);
fabian.la...@symfony-project.com> wrote:
> Hi Thomas,
> I was not able to get your code running easily.
> Here a drop in zip file with my tests. I also noted you were not
> always passing true to memory_get_usage. any reason for that?
> no reason
> To eliminate any chance for error i start with a routing.yml file
> which i generate with 1000 routes (high number but not too
> unrealistic)
> After that I generate the 1.2 and 1.3 version of the confighandler file
> Then i am loading this configuration into pattern routing and running
> a query against them
> I monkeypatched the sfPatternRouting to make them running isolated.
> this should not have any impact on the measurements
> 1.2 uses less memory and is faster on generation. This is
> understandable, because 1.3 moves the route compilation to here.
> loading the config is really interesting. it does'nt take memory for
> 1.2 (i don't know why) and it is faster than in 1.3 (because there is
> less to load, because less was generated). It takes TONS of memory for
> 1.3
> then finding the route is WAY faster in 1.3 than in 1.2. However,
> assuming this simple uri against simple route matching is ok for our
> test, the gain in parse() does not offset the heavy lifting on loading
> the configuration.
> Also the memory consumption is not really nice in 1.3
> @Fabien, any immediate ideas? We have to work on this one still
> Fabian
> On Wed, Oct 28, 2009 at 4:46 PM, Thomas Rabaix <thomas.rab...@gmail.com>
> wrote:
> > Hello,
> > I have done a small benchmark with routing implementation with sf1.2
> > and sf1.3.
> > I have created 4 cache files with 500 routes in each files.
> > - test 1 : symfony 1.2 , routing.yml.php - object creation in an
> > array
> > - test 2 : version "cache" symfony 1.2, serialized array with
> > compiled routes from test 1
> > - test 3 : version swOptimizeRoutesPlugin : array of serialized
> > routes
> > - test 4 : version symfony 1.3, a big array
> > Here the result (I have been running this test with php5.3 / macport):
> > # TEST 1 : sf1.2 / sfPatternRouting - object creation
> > # TEST 1 - file size: 398576.000000, include time: 0.022773, memory:
> > 2544656
> > # TEST 2 : sf1.2 / sfPatternRouting - object deserialization as if
> > cache is enabled
> > # TEST 2 - file size: 398576.000000, include time: 0.032223, memory:
> > 5787200
> > # TEST 3 : sf1.2 / swOptimizePatternRouting - array with serialized
> > route informations
> > # TEST 3 - file size: 426672.000000, include time: 0.004196, memory:
> > 2183944
> > # TEST 4 : sf1.3 / sfPatternRouting like - full array
> > # TEST 4 - file size: 652672.000000, include time: 0.036927, memory:
> > *9573008*
> > My first conclusion is that sf1.3 take more memory than any other
> > implementations without any speed gain. Of course I might missing
> > something important ...
I don't have time today to have a look at the issue, but keep in mind that the speed of the loading is not the whole picture. We also need to have a fast route generation, especially when you need to generate 30-50 links per page. So, we need to find a good balance between loading speed and generation speed.
As far as mod_rewrite is concerned, this won't work as the symfony routing is more than just pattern matching.
Thomas Rabaix wrote:
> I am glad that you get the time to look at this issue. I don't know if > you have a look into the plugin, but I don't think symfony or either PHP > can deal with this work : loading this information into memory is too > slow : 304ms to load 1000 routes (maybe it will be 90ms for 200 routes).
> This logic can be easily ported into the webserver's configuration > (maybe only in production) with mod_rewrite, so the matching is not done > at the symfony/php level.
> The next thing, route information should be loaded on demand [1], each > page can have an average of 20-30 routes loaded (why the other routes > should be loaded ?). But there is another problem, PHP is slow to > deserialize string, so loading on demand routes will be slow...
> Now, I am pretty confident that the native php serializer is not > suitable for routing, maybe the sfRoute should have its own > serializer/deserializer. So in this case loading routes can be quicker. > I have not try this because this will require to patch symfony, I don't > want to propose this as a plugin.
> [1] on demand : the cache system only hold an array of serialized routes :
> array(
> 'my_route_1' => array('sfDoctrineRoute', "serialized data"),
> 'my_route_2' => array('sfDoctrineRoute', "serialized data"),
> [...]
> );
> On Wed, Oct 28, 2009 at 10:58 PM, Fabian Lange > <fabian.la...@symfony-project.com > <mailto:fabian.la...@symfony-project.com>> wrote:
> Hi Thomas,
> I was not able to get your code running easily.
> Here a drop in zip file with my tests. I also noted you were not
> always passing true to memory_get_usage. any reason for that?
> no reason
> To eliminate any chance for error i start with a routing.yml file
> which i generate with 1000 routes (high number but not too
> unrealistic)
> After that I generate the 1.2 and 1.3 version of the confighandler file
> Then i am loading this configuration into pattern routing and running
> a query against them
> I monkeypatched the sfPatternRouting to make them running isolated.
> this should not have any impact on the measurements
> 1.2 uses less memory and is faster on generation. This is
> understandable, because 1.3 moves the route compilation to here.
> loading the config is really interesting. it does'nt take memory for
> 1.2 (i don't know why) and it is faster than in 1.3 (because there is
> less to load, because less was generated). It takes TONS of memory for
> 1.3
> then finding the route is WAY faster in 1.3 than in 1.2. However,
> assuming this simple uri against simple route matching is ok for our
> test, the gain in parse() does not offset the heavy lifting on loading
> the configuration.
> Also the memory consumption is not really nice in 1.3
> @Fabien, any immediate ideas? We have to work on this one still
> Fabian
> On Wed, Oct 28, 2009 at 4:46 PM, Thomas Rabaix
> <thomas.rab...@gmail.com <mailto:thomas.rab...@gmail.com>> wrote:
> > Hello,
> > I have done a small benchmark with routing implementation with sf1.2
> > and sf1.3.
> > I have created 4 cache files with 500 routes in each files.
> > - test 1 : symfony 1.2 , routing.yml.php - object creation in an
> > array
> > - test 2 : version "cache" symfony 1.2, serialized array with
> > compiled routes from test 1
> > - test 3 : version swOptimizeRoutesPlugin : array of serialized
> > routes
> > - test 4 : version symfony 1.3, a big array
> > Here the result (I have been running this test with php5.3 /
> macport):
> > # TEST 1 : sf1.2 / sfPatternRouting - object creation
> > # TEST 1 - file size: 398576.000000, include time: 0.022773, memory:
> > 2544656
> > # TEST 2 : sf1.2 / sfPatternRouting - object deserialization as if
> > cache is enabled
> > # TEST 2 - file size: 398576.000000, include time: 0.032223, memory:
> > 5787200
> > # TEST 3 : sf1.2 / swOptimizePatternRouting - array with serialized
> > route informations
> > # TEST 3 - file size: 426672.000000, include time: 0.004196, memory:
> > 2183944
> > # TEST 4 : sf1.3 / sfPatternRouting like - full array
> > # TEST 4 - file size: 652672.000000, include time: 0.036927, memory:
> > *9573008*
> > My first conclusion is that sf1.3 take more memory than any other
> > implementations without any speed gain. Of course I might missing
> > something important ...
The plugin is just a proof of concept, but the apache mod_rewrite version is
working fine. The mod_rewrite just add a new variable 'sf_route' so the
sfRouting does not have to find the route. Of course, a custom sfRoute
object can add more logic into the "match" method. But this should not be
advocated, just see the number... finding a route with regular expression is
"slow" and if the dev add another extra layer of check, this will be _very_
slow ...
On Thu, Oct 29, 2009 at 9:17 AM, Fabien Potencier <
> I don't have time today to have a look at the issue, but keep in mind
> that the speed of the loading is not the whole picture. We also need to
> have a fast route generation, especially when you need to generate 30-50
> links per page. So, we need to find a good balance between loading speed
> and generation speed.
> As far as mod_rewrite is concerned, this won't work as the symfony
> routing is more than just pattern matching.
> Thomas Rabaix wrote:
> > I am glad that you get the time to look at this issue. I don't know if
> > you have a look into the plugin, but I don't think symfony or either PHP
> > can deal with this work : loading this information into memory is too
> > slow : 304ms to load 1000 routes (maybe it will be 90ms for 200 routes).
> > This logic can be easily ported into the webserver's configuration
> > (maybe only in production) with mod_rewrite, so the matching is not done
> > at the symfony/php level.
> > The next thing, route information should be loaded on demand [1], each
> > page can have an average of 20-30 routes loaded (why the other routes
> > should be loaded ?). But there is another problem, PHP is slow to
> > deserialize string, so loading on demand routes will be slow...
> > Now, I am pretty confident that the native php serializer is not
> > suitable for routing, maybe the sfRoute should have its own
> > serializer/deserializer. So in this case loading routes can be quicker.
> > I have not try this because this will require to patch symfony, I don't
> > want to propose this as a plugin.
> > [1] on demand : the cache system only hold an array of serialized routes
> :
> > array(
> > 'my_route_1' => array('sfDoctrineRoute', "serialized data"),
> > 'my_route_2' => array('sfDoctrineRoute', "serialized data"),
> > [...]
> > );
> > On Wed, Oct 28, 2009 at 10:58 PM, Fabian Lange
> > <fabian.la...@symfony-project.com
> > <mailto:fabian.la...@symfony-project.com>> wrote:
> > Hi Thomas,
> > I was not able to get your code running easily.
> > Here a drop in zip file with my tests. I also noted you were not
> > always passing true to memory_get_usage. any reason for that?
> > no reason
> > To eliminate any chance for error i start with a routing.yml file
> > which i generate with 1000 routes (high number but not too
> > unrealistic)
> > After that I generate the 1.2 and 1.3 version of the confighandler
> file
> > Then i am loading this configuration into pattern routing and running
> > a query against them
> > I monkeypatched the sfPatternRouting to make them running isolated.
> > this should not have any impact on the measurements
> > 1.2 uses less memory and is faster on generation. This is
> > understandable, because 1.3 moves the route compilation to here.
> > loading the config is really interesting. it does'nt take memory for
> > 1.2 (i don't know why) and it is faster than in 1.3 (because there is
> > less to load, because less was generated). It takes TONS of memory
> for
> > 1.3
> > then finding the route is WAY faster in 1.3 than in 1.2. However,
> > assuming this simple uri against simple route matching is ok for our
> > test, the gain in parse() does not offset the heavy lifting on
> loading
> > the configuration.
> > Also the memory consumption is not really nice in 1.3
> > @Fabien, any immediate ideas? We have to work on this one still
> > Fabian
> > On Wed, Oct 28, 2009 at 4:46 PM, Thomas Rabaix
> > <thomas.rab...@gmail.com <mailto:thomas.rab...@gmail.com>> wrote:
> > > Hello,
> > > I have done a small benchmark with routing implementation with
> sf1.2
> > > and sf1.3.
> > > I have created 4 cache files with 500 routes in each files.
> > > - test 1 : symfony 1.2 , routing.yml.php - object creation in an
> > > array
> > > - test 2 : version "cache" symfony 1.2, serialized array with
> > > compiled routes from test 1
> > > - test 3 : version swOptimizeRoutesPlugin : array of serialized
> > > routes
> > > - test 4 : version symfony 1.3, a big array
> > > Here the result (I have been running this test with php5.3 /
> > macport):
> > > # TEST 1 : sf1.2 / sfPatternRouting - object creation
> > > # TEST 1 - file size: 398576.000000, include time: 0.022773,
> memory:
> > > 2544656
> > > # TEST 2 : sf1.2 / sfPatternRouting - object deserialization as
> if
> > > cache is enabled
> > > # TEST 2 - file size: 398576.000000, include time: 0.032223,
> memory:
> > > 5787200
> > > # TEST 3 : sf1.2 / swOptimizePatternRouting - array with
> serialized
> > > route informations
> > > # TEST 3 - file size: 426672.000000, include time: 0.004196,
> memory:
> > > 2183944
> > > # TEST 4 : sf1.3 / sfPatternRouting like - full array
> > > # TEST 4 - file size: 652672.000000, include time: 0.036927,
> memory:
> > > *9573008*
> > > My first conclusion is that sf1.3 take more memory than any other
> > > implementations without any speed gain. Of course I might missing
> > > something important ...
Thomas Rabaix wrote:
> The plugin is just a proof of concept, but the apache mod_rewrite > version is working fine. The mod_rewrite just add a new variable > 'sf_route' so the sfRouting does not have to find the route. Of course, > a custom sfRoute object can add more logic into the "match" method. But > this should not be advocated, just see the number... finding a route > with regular expression is "slow" and if the dev add another extra layer > of check, this will be _very_ slow ...
So, basically, you want to remove all the advanced features of the symfony routing? I fail to understand how a mod_rewrite configuration can "replace" the actual routing.
Also, keep in mind that we also need something fast to generate a route.
> On Thu, Oct 29, 2009 at 9:17 AM, Fabien Potencier > <fabien.potenc...@symfony-project.com > <mailto:fabien.potenc...@symfony-project.com>> wrote:
> I don't have time today to have a look at the issue, but keep in mind
> that the speed of the loading is not the whole picture. We also need to
> have a fast route generation, especially when you need to generate 30-50
> links per page. So, we need to find a good balance between loading speed
> and generation speed.
> As far as mod_rewrite is concerned, this won't work as the symfony
> routing is more than just pattern matching.
> Thomas Rabaix wrote:
> > I am glad that you get the time to look at this issue. I don't
> know if
> > you have a look into the plugin, but I don't think symfony or
> either PHP
> > can deal with this work : loading this information into memory is too
> > slow : 304ms to load 1000 routes (maybe it will be 90ms for 200
> routes).
> > This logic can be easily ported into the webserver's configuration
> > (maybe only in production) with mod_rewrite, so the matching is
> not done
> > at the symfony/php level.
> > The next thing, route information should be loaded on demand [1],
> each
> > page can have an average of 20-30 routes loaded (why the other routes
> > should be loaded ?). But there is another problem, PHP is slow to
> > deserialize string, so loading on demand routes will be slow...
> > Now, I am pretty confident that the native php serializer is not
> > suitable for routing, maybe the sfRoute should have its own
> > serializer/deserializer. So in this case loading routes can be
> quicker.
> > I have not try this because this will require to patch symfony, I
> don't
> > want to propose this as a plugin.
> > [1] on demand : the cache system only hold an array of serialized
> routes :
> > array(
> > 'my_route_1' => array('sfDoctrineRoute', "serialized data"),
> > 'my_route_2' => array('sfDoctrineRoute', "serialized data"),
> > [...]
> > );
> > On Wed, Oct 28, 2009 at 10:58 PM, Fabian Lange
> > <fabian.la...@symfony-project.com
> <mailto:fabian.la...@symfony-project.com>
> > <mailto:fabian.la...@symfony-project.com
> <mailto:fabian.la...@symfony-project.com>>> wrote:
> > Hi Thomas,
> > I was not able to get your code running easily.
> > Here a drop in zip file with my tests. I also noted you were not
> > always passing true to memory_get_usage. any reason for that?
> > no reason
> > To eliminate any chance for error i start with a routing.yml file
> > which i generate with 1000 routes (high number but not too
> > unrealistic)
> > After that I generate the 1.2 and 1.3 version of the
> confighandler file
> > Then i am loading this configuration into pattern routing and
> running
> > a query against them
> > I monkeypatched the sfPatternRouting to make them running
> isolated.
> > this should not have any impact on the measurements
> > 1.2 uses less memory and is faster on generation. This is
> > understandable, because 1.3 moves the route compilation to here.
> > loading the config is really interesting. it does'nt take
> memory for
> > 1.2 (i don't know why) and it is faster than in 1.3 (because
> there is
> > less to load, because less was generated). It takes TONS of
> memory for
> > 1.3
> > then finding the route is WAY faster in 1.3 than in 1.2. However,
> > assuming this simple uri against simple route matching is ok
> for our
> > test, the gain in parse() does not offset the heavy lifting
> on loading
> > the configuration.
> > Also the memory consumption is not really nice in 1.3
> > @Fabien, any immediate ideas? We have to work on this one still
> > Fabian
> > On Wed, Oct 28, 2009 at 4:46 PM, Thomas Rabaix
> > <thomas.rab...@gmail.com <mailto:thomas.rab...@gmail.com>
> <mailto:thomas.rab...@gmail.com <mailto:thomas.rab...@gmail.com>>>
> wrote:
> > > Hello,
> > > I have done a small benchmark with routing implementation
> with sf1.2
> > > and sf1.3.
> > > I have created 4 cache files with 500 routes in each files.
> > > - test 1 : symfony 1.2 , routing.yml.php - object
> creation in an
> > > array
> > > - test 2 : version "cache" symfony 1.2, serialized array
> with
> > > compiled routes from test 1
> > > - test 3 : version swOptimizeRoutesPlugin : array of
> serialized
> > > routes
> > > - test 4 : version symfony 1.3, a big array
> > > Here the result (I have been running this test with php5.3 /
> > macport):
> > > # TEST 1 : sf1.2 / sfPatternRouting - object creation
> > > # TEST 1 - file size: 398576.000000, include time:
> 0.022773, memory:
> > > 2544656
> > > # TEST 2 : sf1.2 / sfPatternRouting - object
> deserialization as if
> > > cache is enabled
> > > # TEST 2 - file size: 398576.000000, include time:
> 0.032223, memory:
> > > 5787200
> > > # TEST 3 : sf1.2 / swOptimizePatternRouting - array with
> serialized
> > > route informations
> > > # TEST 3 - file size: 426672.000000, include time:
> 0.004196, memory:
> > > 2183944
> > > # TEST 4 : sf1.3 / sfPatternRouting like - full array
> > > # TEST 4 - file size: 652672.000000, include time:
> 0.036927, memory:
> > > *9573008*
> > > My first conclusion is that sf1.3 take more memory than
> any other
> > > implementations without any speed gain. Of course I might
> missing
> > > something important ...
> Thomas Rabaix wrote: >> The plugin is just a proof of concept, but the apache mod_rewrite >> version is working fine. The mod_rewrite just add a new variable >> 'sf_route' so the sfRouting does not have to find the route. Of >> course, >> a custom sfRoute object can add more logic into the "match" method. >> But >> this should not be advocated, just see the number... finding a route >> with regular expression is "slow" and if the dev add another extra >> layer >> of check, this will be _very_ slow ...
> So, basically, you want to remove all the advanced features of the > symfony routing? I fail to understand how a mod_rewrite configuration > can "replace" the actual routing.
> Also, keep in mind that we also need something fast to generate a > route.
Well I do not know what the yahoo guys did in the end, but dealing with lots of different routes is one of the key performance issues from what I have seen. So it makes sense to allow people to be able to replace it with more streamlined approaches if these are sufficient for them. However the default should stick to the full feature solution imho .. not only for BC but also because most people do not really suffer much at all from the overhead that the routing system brings.
Does the advance features deal with route matching ? the mod_rewrite will
not replace the routing, mod_rewrite is only used to quickly find the route.
That's all [1]
swOptimizeRoutesPlugin is just a proof of concept, And it will break routes
which are using a custom sfRoute::match method. I am pretty sure that should
concern only 0.1% of symfony users, and these users will have enough
experiences to not used the plugin !
Now the original idea of the post was to show that the current
implementation as 1.3 is slow and take a lot of memory. That was my first
concern.
I am agree with Fabian, the routing should be splited into a matcher and a
generater. (and/or maybe use some kind of namespace)
> Thomas Rabaix wrote:
> > The plugin is just a proof of concept, but the apache mod_rewrite
> > version is working fine. The mod_rewrite just add a new variable
> > 'sf_route' so the sfRouting does not have to find the route. Of course,
> > a custom sfRoute object can add more logic into the "match" method. But
> > this should not be advocated, just see the number... finding a route
> > with regular expression is "slow" and if the dev add another extra layer
> > of check, this will be _very_ slow ...
> So, basically, you want to remove all the advanced features of the
> symfony routing? I fail to understand how a mod_rewrite configuration
> can "replace" the actual routing.
> Also, keep in mind that we also need something fast to generate a route.
> Fabien
> > On Thu, Oct 29, 2009 at 9:17 AM, Fabien Potencier
> > <fabien.potenc...@symfony-project.com
> > <mailto:fabien.potenc...@symfony-project.com>> wrote:
> > I don't have time today to have a look at the issue, but keep in mind
> > that the speed of the loading is not the whole picture. We also need
> to
> > have a fast route generation, especially when you need to generate
> 30-50
> > links per page. So, we need to find a good balance between loading
> speed
> > and generation speed.
> > As far as mod_rewrite is concerned, this won't work as the symfony
> > routing is more than just pattern matching.
> > Thomas Rabaix wrote:
> > > I am glad that you get the time to look at this issue. I don't
> > know if
> > > you have a look into the plugin, but I don't think symfony or
> > either PHP
> > > can deal with this work : loading this information into memory is
> too
> > > slow : 304ms to load 1000 routes (maybe it will be 90ms for 200
> > routes).
> > > This logic can be easily ported into the webserver's configuration
> > > (maybe only in production) with mod_rewrite, so the matching is
> > not done
> > > at the symfony/php level.
> > > The next thing, route information should be loaded on demand [1],
> > each
> > > page can have an average of 20-30 routes loaded (why the other
> routes
> > > should be loaded ?). But there is another problem, PHP is slow to
> > > deserialize string, so loading on demand routes will be slow...
> > > Now, I am pretty confident that the native php serializer is not
> > > suitable for routing, maybe the sfRoute should have its own
> > > serializer/deserializer. So in this case loading routes can be
> > quicker.
> > > I have not try this because this will require to patch symfony, I
> > don't
> > > want to propose this as a plugin.
> > > [1] on demand : the cache system only hold an array of serialized
> > routes :
> > > array(
> > > 'my_route_1' => array('sfDoctrineRoute', "serialized data"),
> > > 'my_route_2' => array('sfDoctrineRoute', "serialized data"),
> > > [...]
> > > );
> > > On Wed, Oct 28, 2009 at 10:58 PM, Fabian Lange
> > > <fabian.la...@symfony-project.com
> > <mailto:fabian.la...@symfony-project.com>
> > > <mailto:fabian.la...@symfony-project.com
> > <mailto:fabian.la...@symfony-project.com>>> wrote:
> > > Hi Thomas,
> > > I was not able to get your code running easily.
> > > Here a drop in zip file with my tests. I also noted you were
> not
> > > always passing true to memory_get_usage. any reason for that?
> > > no reason
> > > To eliminate any chance for error i start with a routing.yml
> file
> > > which i generate with 1000 routes (high number but not too
> > > unrealistic)
> > > After that I generate the 1.2 and 1.3 version of the
> > confighandler file
> > > Then i am loading this configuration into pattern routing and
> > running
> > > a query against them
> > > I monkeypatched the sfPatternRouting to make them running
> > isolated.
> > > this should not have any impact on the measurements
> > > 1.2 uses less memory and is faster on generation. This is
> > > understandable, because 1.3 moves the route compilation to
> here.
> > > loading the config is really interesting. it does'nt take
> > memory for
> > > 1.2 (i don't know why) and it is faster than in 1.3 (because
> > there is
> > > less to load, because less was generated). It takes TONS of
> > memory for
> > > 1.3
> > > then finding the route is WAY faster in 1.3 than in 1.2.
> However,
> > > assuming this simple uri against simple route matching is ok
> > for our
> > > test, the gain in parse() does not offset the heavy lifting
> > on loading
> > > the configuration.
> > > Also the memory consumption is not really nice in 1.3
> > > @Fabien, any immediate ideas? We have to work on this one
> still
> > > Fabian
> > > On Wed, Oct 28, 2009 at 4:46 PM, Thomas Rabaix
> > > <thomas.rab...@gmail.com <mailto:thomas.rab...@gmail.com>
> > <mailto:thomas.rab...@gmail.com <mailto:thomas.rab...@gmail.com>>>
> > wrote:
> > > > Hello,
> > > > I have done a small benchmark with routing implementation
> > with sf1.2
> > > > and sf1.3.
> > > > I have created 4 cache files with 500 routes in each files.
> > > > - test 1 : symfony 1.2 , routing.yml.php - object
> > creation in an
> > > > array
> > > > - test 2 : version "cache" symfony 1.2, serialized array
> > with
> > > > compiled routes from test 1
> > > > - test 3 : version swOptimizeRoutesPlugin : array of
> > serialized
> > > > routes
> > > > - test 4 : version symfony 1.3, a big array
> > > > Here the result (I have been running this test with php5.3
> /
> > > macport):
> > > > # TEST 1 : sf1.2 / sfPatternRouting - object creation
> > > > # TEST 1 - file size: 398576.000000, include time:
> > 0.022773, memory:
> > > > 2544656
> > > > # TEST 2 : sf1.2 / sfPatternRouting - object
> > deserialization as if
> > > > cache is enabled
> > > > # TEST 2 - file size: 398576.000000, include time:
> > 0.032223, memory:
> > > > 5787200
> > > > # TEST 3 : sf1.2 / swOptimizePatternRouting - array with
> > serialized
> > > > route informations
> > > > # TEST 3 - file size: 426672.000000, include time:
> > 0.004196, memory:
> > > > 2183944
> > > > # TEST 4 : sf1.3 / sfPatternRouting like - full array
> > > > # TEST 4 - file size: 652672.000000, include time:
> > 0.036927, memory:
> > > > *9573008*
> > > > My first conclusion is that sf1.3 take more memory than
> > any other
> > > > implementations without any speed gain. Of course I might
> > missing
> > > > something important ...
<thomas.rab...@gmail.com> wrote: > Does the advance features deal with route matching ? the mod_rewrite will > not replace the routing, mod_rewrite is only used to quickly find the > route. > That's all [1]
> swOptimizeRoutesPlugin is just a proof of concept, And it will break > routes > which are using a custom sfRoute::match method. I am pretty sure that > should > concern only 0.1% of symfony users, and these users will have enough > experiences to not used the plugin !
> Now the original idea of the post was to show that the current > implementation as 1.3 is slow and take a lot of memory. That was my first > concern.
> I am agree with Fabian, the routing should be splited into a matcher and > a > generater. (and/or maybe use some kind of namespace)
> On Thu, Oct 29, 2009 at 10:35 AM, Fabien Potencier < > fabien.potenc...@symfony-project.com> wrote:
>> Thomas Rabaix wrote: >> > The plugin is just a proof of concept, but the apache mod_rewrite >> > version is working fine. The mod_rewrite just add a new variable >> > 'sf_route' so the sfRouting does not have to find the route. Of >> course, >> > a custom sfRoute object can add more logic into the "match" method. >> But >> > this should not be advocated, just see the number... finding a route >> > with regular expression is "slow" and if the dev add another extra >> layer >> > of check, this will be _very_ slow ...
>> So, basically, you want to remove all the advanced features of the >> symfony routing? I fail to understand how a mod_rewrite configuration >> can "replace" the actual routing.
>> Also, keep in mind that we also need something fast to generate a route.
>> Fabien
>> > On Thu, Oct 29, 2009 at 9:17 AM, Fabien Potencier >> > <fabien.potenc...@symfony-project.com >> > <mailto:fabien.potenc...@symfony-project.com>> wrote:
>> > I don't have time today to have a look at the issue, but keep in >> mind >> > that the speed of the loading is not the whole picture. We also >> need >> to >> > have a fast route generation, especially when you need to generate >> 30-50 >> > links per page. So, we need to find a good balance between loading >> speed >> > and generation speed.
>> > As far as mod_rewrite is concerned, this won't work as the symfony >> > routing is more than just pattern matching.
>> > Thomas Rabaix wrote: >> > > I am glad that you get the time to look at this issue. I don't >> > know if >> > > you have a look into the plugin, but I don't think symfony or >> > either PHP >> > > can deal with this work : loading this information into memory >> is >> too >> > > slow : 304ms to load 1000 routes (maybe it will be 90ms for 200 >> > routes).
>> > > This logic can be easily ported into the webserver's >> configuration >> > > (maybe only in production) with mod_rewrite, so the matching is >> > not done >> > > at the symfony/php level.
>> > > The next thing, route information should be loaded on demand >> [1], >> > each >> > > page can have an average of 20-30 routes loaded (why the other >> routes >> > > should be loaded ?). But there is another problem, PHP is slow >> to >> > > deserialize string, so loading on demand routes will be slow...
>> > > Now, I am pretty confident that the native php serializer is >> not >> > > suitable for routing, maybe the sfRoute should have its own >> > > serializer/deserializer. So in this case loading routes can be >> > quicker. >> > > I have not try this because this will require to patch >> symfony, I >> > don't >> > > want to propose this as a plugin.
>> > > [1] on demand : the cache system only hold an array of >> serialized >> > routes : >> > > array( >> > > 'my_route_1' => array('sfDoctrineRoute', "serialized data"), >> > > 'my_route_2' => array('sfDoctrineRoute', "serialized data"), >> > > [...] >> > > );
>> > > On Wed, Oct 28, 2009 at 10:58 PM, Fabian Lange >> > > <fabian.la...@symfony-project.com >> > <mailto:fabian.la...@symfony-project.com> >> > > <mailto:fabian.la...@symfony-project.com >> > <mailto:fabian.la...@symfony-project.com>>> wrote:
>> > > Hi Thomas,
>> > > I was not able to get your code running easily. >> > > Here a drop in zip file with my tests. I also noted you >> were >> not >> > > always passing true to memory_get_usage. any reason for >> that?
>> > > no reason
>> > > To eliminate any chance for error i start with a >> routing.yml >> file >> > > which i generate with 1000 routes (high number but not too >> > > unrealistic) >> > > After that I generate the 1.2 and 1.3 version of the >> > confighandler file >> > > Then i am loading this configuration into pattern routing >> and >> > running >> > > a query against them >> > > I monkeypatched the sfPatternRouting to make them running >> > isolated. >> > > this should not have any impact on the measurements
>> > > 1.2 uses less memory and is faster on generation. This is >> > > understandable, because 1.3 moves the route compilation to >> here.
>> > > loading the config is really interesting. it does'nt take >> > memory for >> > > 1.2 (i don't know why) and it is faster than in 1.3 >> (because >> > there is >> > > less to load, because less was generated). It takes TONS of >> > memory for >> > > 1.3
>> > > then finding the route is WAY faster in 1.3 than in 1.2. >> However, >> > > assuming this simple uri against simple route matching is >> ok >> > for our >> > > test, the gain in parse() does not offset the heavy lifting >> > on loading >> > > the configuration. >> > > Also the memory consumption is not really nice in 1.3
>> > > @Fabien, any immediate ideas? We have to work on this one >> still
>> > > Fabian
>> > > On Wed, Oct 28, 2009 at 4:46 PM, Thomas Rabaix >> > > <thomas.rab...@gmail.com <mailto:thomas.rab...@gmail.com> >> > <mailto:thomas.rab...@gmail.com <mailto:thomas.rab...@gmail.com>>> >> > wrote:
>> > > > Hello,
>> > > > I have done a small benchmark with routing >> implementation >> > with sf1.2 >> > > > and sf1.3.
>> > > > I have created 4 cache files with 500 routes in each >> files. >> > > > - test 1 : symfony 1.2 , routing.yml.php - object >> > creation in an >> > > > array >> > > > - test 2 : version "cache" symfony 1.2, serialized >> array >> > with >> > > > compiled routes from test 1 >> > > > - test 3 : version swOptimizeRoutesPlugin : array of >> > serialized >> > > > routes >> > > > - test 4 : version symfony 1.3, a big array
>> > > > Here the result (I have been running this test with >> php5.3 >> / >> > > macport):
>> > > > # TEST 1 : sf1.2 / sfPatternRouting - object creation >> > > > # TEST 1 - file size: 398576.000000, include time: >> > 0.022773, memory: >> > > > 2544656 >> > > > # TEST 2 : sf1.2 / sfPatternRouting - object >> > deserialization as if >> > > > cache is enabled >> > > > # TEST 2 - file size: 398576.000000, include time: >> > 0.032223, memory: >> > > > 5787200 >> > > > # TEST 3 : sf1.2 / swOptimizePatternRouting - array with >> > serialized >> > > > route informations >> > > > # TEST 3 - file size: 426672.000000, include time: >> > 0.004196, memory: >> > > > 2183944 >> > > > # TEST 4 : sf1.3 / sfPatternRouting like - full array >> > > > # TEST 4 - file size:
I added a version from 1.2, your proposal and my patch.
I would appreciate if some more people could help me with the stuff in Todo: - create a realistic routing.yml generator - create a realistig usage of the files (loadConfiguration/parse/findurl/generate....)
I think there is one important point open for discussion: What is the difference between generated routing config file and the cache file. i think the difference blurs, but the cache is configurable (apc memcache, db... ) while the config file is not. Putting the full routes into the config files looks exceptionally wrong for me. But if it helps we should consider dropping the cache support (which caused only pain the last months)
Cool - I'll take a look.
Would be handy if there was a resource available that allowed people to
post their routing files - the more live/"real world" data available -
the better. I'm not sure people would like to post their routing files
to the general group...
The first version I sent was really just a proof of concept - further
refinement significantly improves matters. The current version does a
very simple comparison of arrays - which has since been improved.
Further improvement will come from issue you mentioned - regarding the
config file & cache.
I think there's a number of way of solving the problem in a way that
degrades gracefully from an optimal configuration - and hence would be
usable generally.
Currently metrics tends to show performance bottlenecks in other areas -
rather than with the routing side of things.
Locally I did extend the final test part of the "routing performance" to
include random routes and not-found routes. I did also find that
running 2 sf 1.3 config file generators without destroying the
routingConfigHandler caused the subsequent tests to abort.
> I added a version from 1.2, your proposal and my patch.
> I would appreciate if some more people could help me with the stuff in Todo:
> - create a realistic routing.yml generator
> - create a realistig usage of the files
> (loadConfiguration/parse/findurl/generate....)
> I think there is one important point open for discussion:
> What is the difference between generated routing config file and the cache file.
> i think the difference blurs, but the cache is configurable (apc
> memcache, db... ) while the config file is not.
> Putting the full routes into the config files looks exceptionally
> wrong for me. But if it helps we should consider dropping the cache
> support (which caused only pain the last months)
> Fabian
> On Thu, Oct 29, 2009 at 1:50 PM, david <da...@inspiredthinking.co.uk> wrote:
> > While it's rough around the edges and can be enhanced a lot more -
> > attached is an extension of the test framework that Fabien L mailed
> > earlier.
> > With the 13a version:
> > 37% reduction in mem usage
> > 30% improvement in speed
> > 40% improvement in the generated route cache file size
> > If you're interested I also have some real performance metrics available.
Hi David, feel free to send me patches for our performance test suite. I will incorporate them until i find a way to give access to everybody.
So if you have something new i will include it.
Ideally we have then some kind of incubation of a new way for routing to be configured. And we can benchmark them.
So that before release 1.3 we can find a good performing solution. To be included in 1.3 i would like to accept only "small" backwards compatible changes that have a big net effect.
Here is output from my run of the tests - I cannot run david's
benchmark as splFixedArray is not available in my php build, though
SPL is shown as available in php_info(). I'm running php 5.2.11.
Re: config/cache - in 1.3 our config_routing.yml.php is 2,054,492
bytes and consists of 628 routes. These are mostly propel routes.
However, with APC opcode caching we find this loads very quickly (last
time I benchmarked it was something like 0.06 seconds) Our problem is
matching routes - if we have a page with 50 or so links, the majority
of the time is spent matching the route (something like 1 second out
of total request time of 1.2 seconds) If we enable the APC variable
cache and lookup_dedicated_cache_keys once the route is matched, the
performance is ok, however we have a lot of route variations and I can
see this using a large amount of RAM. We cannot run without this cache
as performance is just too slow. Been a while since I looked at this,
but can put together some stats if it's useful?
[andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
symfony/test/routing_performance% php routing_performance.php 1.2|grep
"^#"
# testing patch 1.2
# This is equivalent to the latest version from the 1.2 branch.
# Creating routing yml with 1500+ routes
# 1.2 Routing - Creating Config
# time: 1.713294, memory: 2304.00kb
# 1.2 Routing - Loading Config
# time: 0.193375, memory: 10240.00kb
# 1.2 Routing - Searching Routes 10 times
# 1.2 Routing - Generating Routes 20 times
# time: 1.633093, memory: 3328.00kb
# 1.2 Routing - Real World Test - Accumulated data
# time: 1.826847, memory: 13568.00kb
# file: 211.00kb
[andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
symfony/test/routing_performance% php routing_performance.php david|
grep "^#"
# testing patch david
# Creating routing yml with 1500+ routes
# david Routing - Creating Config
PHP Fatal error: Class 'splFixedArray' not found in /home/andyr/
projects/trunk-local-fs/lib/vendor/symfony/test/routing_performance/
david/sfRoutingConfigHandlerNew.class.php on line 84
[andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
symfony/test/routing_performance% php routing_performance.php fabian|
grep "^#"
# testing patch fabian
# Creating routing yml with 1500+ routes
# fabian Routing - Creating Config
# time: 2.712385, memory: 9472.00kb
# fabian Routing - Loading Config
# time: 0.421317, memory: 9216.00kb
# fabian Routing - Searching Routes 10 times
# fabian Routing - Generating Routes 20 times
# time: 0.814632, memory: 0.00kb
# fabian Routing - Real World Test - Accumulated data
# time: 1.236377, memory: 9216.00kb
# file: 1899.08kb
Thanks,
Andy.
On Oct 29, 5:14 pm, Fabian Lange <fabian.la...@symfony-project.com>
wrote:
> Hi David,
> feel free to send me patches for our performance test suite.
> I will incorporate them until i find a way to give access to everybody.
> So if you have something new i will include it.
> Ideally we have then some kind of incubation of a new way for routing
> to be configured. And we can benchmark them.
> So that before release 1.3 we can find a good performing solution.
> To be included in 1.3 i would like to accept only "small" backwards
> compatible changes that have a big net effect.
I ran some tests last night on the old performance framework before
downloading the latest version and starting to migrate my changes into
the code - this is only the compression based changes - nothing
intelligent happening yet:
This is only from targeting compression - I've not looked at any kind
of intelligent mapping/loading yet. The splArray out of habit (I've
been working on a project recently where the splDatastructures are
used). I'll ensure the next version is compatible.
How are you using/how have you configured the APC with sf?
I ask as I've been playing with some autodetection code - the intention
being to use it if it's available - otherwise to fall back to more
conventional approaches.
On Fri, 2009-10-30 at 05:37 -0700, arri...@gmail.com wrote: > Hi,
> Here is output from my run of the tests - I cannot run david's
> benchmark as splFixedArray is not available in my php build, though
> SPL is shown as available in php_info(). I'm running php 5.2.11.
> Re: config/cache - in 1.3 our config_routing.yml.php is 2,054,492
> bytes and consists of 628 routes. These are mostly propel routes.
> However, with APC opcode caching we find this loads very quickly (last
> time I benchmarked it was something like 0.06 seconds) Our problem is
> matching routes - if we have a page with 50 or so links, the majority
> of the time is spent matching the route (something like 1 second out
> of total request time of 1.2 seconds) If we enable the APC variable
> cache and lookup_dedicated_cache_keys once the route is matched, the
> performance is ok, however we have a lot of route variations and I can
> see this using a large amount of RAM. We cannot run without this cache
> as performance is just too slow. Been a while since I looked at this,
> but can put together some stats if it's useful?
> [andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
> symfony/test/routing_performance% php routing_performance.php 1.2|grep
> "^#"
> # testing patch 1.2
> # This is equivalent to the latest version from the 1.2 branch.
> # Creating routing yml with 1500+ routes
> # 1.2 Routing - Creating Config
> # time: 1.713294, memory: 2304.00kb
> # 1.2 Routing - Loading Config
> # time: 0.193375, memory: 10240.00kb
> # 1.2 Routing - Searching Routes 10 times
> # 1.2 Routing - Generating Routes 20 times
> # time: 1.633093, memory: 3328.00kb
> # 1.2 Routing - Real World Test - Accumulated data
> # time: 1.826847, memory: 13568.00kb
> # file: 211.00kb
> [andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
> symfony/test/routing_performance% php routing_performance.php david|
> grep "^#"
> # testing patch david
> # Creating routing yml with 1500+ routes
> # david Routing - Creating Config
> PHP Fatal error: Class 'splFixedArray' not found in /home/andyr/
> projects/trunk-local-fs/lib/vendor/symfony/test/routing_performance/
> david/sfRoutingConfigHandlerNew.class.php on line 84
> [andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
> symfony/test/routing_performance% php routing_performance.php fabian|
> grep "^#"
> # testing patch fabian
> # Creating routing yml with 1500+ routes
> # fabian Routing - Creating Config
> # time: 2.712385, memory: 9472.00kb
> # fabian Routing - Loading Config
> # time: 0.421317, memory: 9216.00kb
> # fabian Routing - Searching Routes 10 times
> # fabian Routing - Generating Routes 20 times
> # time: 0.814632, memory: 0.00kb
> # fabian Routing - Real World Test - Accumulated data
> # time: 1.236377, memory: 9216.00kb
> # file: 1899.08kb
> Thanks,
> Andy.
> On Oct 29, 5:14 pm, Fabian Lange <fabian.la...@symfony-project.com>
> wrote:
> > Hi David,
> > feel free to send me patches for our performance test suite.
> > I will incorporate them until i find a way to give access to everybody.
> > So if you have something new i will include it.
> > Ideally we have then some kind of incubation of a new way for routing
> > to be configured. And we can benchmark them.
> > So that before release 1.3 we can find a good performing solution.
> > To be included in 1.3 i would like to accept only "small" backwards
> > compatible changes that have a big net effect.
We use APC as the opcode cache, so all *.php files are cached by
default - we had to increase the default apc.max_file_size to alow it
to cache the routing cache file (config_routing.yml.php)
We also use the variable cache of APC to speed up routing matches. We
have this entry in factories.yml under routing:
> I ran some tests last night on the old performance framework before
> downloading the latest version and starting to migrate my changes into
> the code - this is only the compression based changes - nothing
> intelligent happening yet:
> This is only from targeting compression - I've not looked at any kind
> of intelligent mapping/loading yet. The splArray out of habit (I've
> been working on a project recently where the splDatastructures are
> used). I'll ensure the next version is compatible.
> How are you using/how have you configured the APC with sf?
> I ask as I've been playing with some autodetection code - the intention
> being to use it if it's available - otherwise to fall back to more
> conventional approaches.
> On Fri, 2009-10-30 at 05:37 -0700, arri...@gmail.com wrote:
> > Hi,
> > Here is output from my run of the tests - I cannot run david's
> > benchmark as splFixedArray is not available in my php build, though
> > SPL is shown as available in php_info(). I'm running php 5.2.11.
> > Re: config/cache - in 1.3 our config_routing.yml.php is 2,054,492
> > bytes and consists of 628 routes. These are mostly propel routes.
> > However, with APC opcode caching we find this loads very quickly (last
> > time I benchmarked it was something like 0.06 seconds) Our problem is
> > matching routes - if we have a page with 50 or so links, the majority
> > of the time is spent matching the route (something like 1 second out
> > of total request time of 1.2 seconds) If we enable the APC variable
> > cache and lookup_dedicated_cache_keys once the route is matched, the
> > performance is ok, however we have a lot of route variations and I can
> > see this using a large amount of RAM. We cannot run without this cache
> > as performance is just too slow. Been a while since I looked at this,
> > but can put together some stats if it's useful?
> > [andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
> > symfony/test/routing_performance% php routing_performance.php 1.2|grep
> > "^#"
> > # testing patch 1.2
> > # This is equivalent to the latest version from the 1.2 branch.
> > # Creating routing yml with 1500+ routes
> > # 1.2 Routing - Creating Config
> > # time: 1.713294, memory: 2304.00kb
> > # 1.2 Routing - Loading Config
> > # time: 0.193375, memory: 10240.00kb
> > # 1.2 Routing - Searching Routes 10 times
> > # 1.2 Routing - Generating Routes 20 times
> > # time: 1.633093, memory: 3328.00kb
> > # 1.2 Routing - Real World Test - Accumulated data
> > # time: 1.826847, memory: 13568.00kb
> > # file: 211.00kb
> > [andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
> > symfony/test/routing_performance% php routing_performance.php david|
> > grep "^#"
> > # testing patch david
> > # Creating routing yml with 1500+ routes
> > # david Routing - Creating Config
> > PHP Fatal error: Class 'splFixedArray' not found in /home/andyr/
> > projects/trunk-local-fs/lib/vendor/symfony/test/routing_performance/
> > david/sfRoutingConfigHandlerNew.class.php on line 84
> > [andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
> > symfony/test/routing_performance% php routing_performance.php fabian|
> > grep "^#"
> > # testing patch fabian
> > # Creating routing yml with 1500+ routes
> > # fabian Routing - Creating Config
> > # time: 2.712385, memory: 9472.00kb
> > # fabian Routing - Loading Config
> > # time: 0.421317, memory: 9216.00kb
> > # fabian Routing - Searching Routes 10 times
> > # fabian Routing - Generating Routes 20 times
> > # time: 0.814632, memory: 0.00kb
> > # fabian Routing - Real World Test - Accumulated data
> > # time: 1.236377, memory: 9216.00kb
> > # file: 1899.08kb
> > Thanks,
> > Andy.
> > On Oct 29, 5:14 pm, Fabian Lange <fabian.la...@symfony-project.com>
> > wrote:
> > > Hi David,
> > > feel free to send me patches for our performance test suite.
> > > I will incorporate them until i find a way to give access to everybody.
> > > So if you have something new i will include it.
> > > Ideally we have then some kind of incubation of a new way for routing
> > > to be configured. And we can benchmark them.
> > > So that before release 1.3 we can find a good performing solution.
> > > To be included in 1.3 i would like to accept only "small" backwards
> > > compatible changes that have a big net effect.
> We use APC as the opcode cache, so all *.php files are cached by
> default - we had to increase the default apc.max_file_size to alow it
> to cache the routing cache file (config_routing.yml.php)
> We also use the variable cache of APC to speed up routing matches. We
> have this entry in factories.yml under routing:
> As I said, the opcode cache pretty much solves the route loading speed
> for us, our issue is the route matching speed without the above cache
> setting.
> On Oct 30, 12:49 pm, David Ashwood <da...@inspiredthinking.co.uk>
> wrote:
> > I ran some tests last night on the old performance framework before
> > downloading the latest version and starting to migrate my changes into
> > the code - this is only the compression based changes - nothing
> > intelligent happening yet:
> > This is only from targeting compression - I've not looked at any kind
> > of intelligent mapping/loading yet. The splArray out of habit (I've
> > been working on a project recently where the splDatastructures are
> > used). I'll ensure the next version is compatible.
> > How are you using/how have you configured the APC with sf?
> > I ask as I've been playing with some autodetection code - the intention
> > being to use it if it's available - otherwise to fall back to more
> > conventional approaches.
> > On Fri, 2009-10-30 at 05:37 -0700, arri...@gmail.com wrote:
> > > Hi,
> > > Here is output from my run of the tests - I cannot run david's
> > > benchmark as splFixedArray is not available in my php build, though
> > > SPL is shown as available in php_info(). I'm running php 5.2.11.
> > > Re: config/cache - in 1.3 our config_routing.yml.php is 2,054,492
> > > bytes and consists of 628 routes. These are mostly propel routes.
> > > However, with APC opcode caching we find this loads very quickly (last
> > > time I benchmarked it was something like 0.06 seconds) Our problem is
> > > matching routes - if we have a page with 50 or so links, the majority
> > > of the time is spent matching the route (something like 1 second out
> > > of total request time of 1.2 seconds) If we enable the APC variable
> > > cache and lookup_dedicated_cache_keys once the route is matched, the
> > > performance is ok, however we have a lot of route variations and I can
> > > see this using a large amount of RAM. We cannot run without this cache
> > > as performance is just too slow. Been a while since I looked at this,
> > > but can put together some stats if it's useful?
> > > On Oct 29, 5:14 pm, Fabian Lange <fabian.la...@symfony-project.com>
> > > wrote:
> > > > Hi David,
> > > > feel free to send me patches for our performance test suite.
> > > > I will incorporate them until i find a way to give access to everybody.
> > > > So if you have something new i will include it.
> > > > Ideally we have then some kind of incubation of a new way for routing
> > > > to be configured. And we can benchmark them.
> > > > So that before release 1.3 we can find a good performing solution.
> > > > To be included in 1.3 i would like to accept only "small" backwards
> > > > compatible changes that have a big net effect.
> all the calls to url_for() collectively take 2,882,401 microseconds
> ~93% of total request time.
> On Oct 30, 2:40 pm, "arri...@gmail.com" <arri...@gmail.com> wrote:
>> Hi,
>> We use APC as the opcode cache, so all *.php files are cached by
>> default - we had to increase the default apc.max_file_size to alow it
>> to cache the routing cache file (config_routing.yml.php)
>> We also use the variable cache of APC to speed up routing matches. We
>> have this entry in factories.yml under routing:
>> As I said, the opcode cache pretty much solves the route loading
>> speed
>> for us, our issue is the route matching speed without the above cache
>> setting.
>> On Oct 30, 12:49 pm, David Ashwood <da...@inspiredthinking.co.uk>
>> wrote:
>>> I ran some tests last night on the old performance framework before
>>> downloading the latest version and starting to migrate my changes
>>> into
>>> the code - this is only the compression based changes - nothing
>>> intelligent happening yet:
>>> This is only from targeting compression - I've not looked at any
>>> kind
>>> of intelligent mapping/loading yet. The splArray out of habit (I've
>>> been working on a project recently where the splDatastructures are
>>> used). I'll ensure the next version is compatible.
>>> How are you using/how have you configured the APC with sf?
>>> I ask as I've been playing with some autodetection code - the
>>> intention
>>> being to use it if it's available - otherwise to fall back to more
>>> conventional approaches.
>>> On Fri, 2009-10-30 at 05:37 -0700, arri...@gmail.com wrote:
>>>> Hi,
>>>> Here is output from my run of the tests - I cannot run david's
>>>> benchmark as splFixedArray is not available in my php build, though
>>>> SPL is shown as available in php_info(). I'm running php 5.2.11.
>>>> Re: config/cache - in 1.3 our config_routing.yml.php is 2,054,492
>>>> bytes and consists of 628 routes. These are mostly propel routes.
>>>> However, with APC opcode caching we find this loads very quickly
>>>> (last
>>>> time I benchmarked it was something like 0.06 seconds) Our
>>>> problem is
>>>> matching routes - if we have a page with 50 or so links, the
>>>> majority
>>>> of the time is spent matching the route (something like 1 second
>>>> out
>>>> of total request time of 1.2 seconds) If we enable the APC variable
>>>> cache and lookup_dedicated_cache_keys once the route is matched,
>>>> the
>>>> performance is ok, however we have a lot of route variations and
>>>> I can
>>>> see this using a large amount of RAM. We cannot run without this
>>>> cache
>>>> as performance is just too slow. Been a while since I looked at
>>>> this,
>>>> but can put together some stats if it's useful?
>>>> [andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
>>>> symfony/test/routing_performance% php routing_performance.php 1.2| >>>> grep
>>>> "^#"
>>>> # testing patch 1.2
>>>> # This is equivalent to the latest version from the 1.2 branch.
>>>> # Creating routing yml with 1500+ routes
>>>> # 1.2 Routing - Creating Config
>>>> # time: 1.713294, memory: 2304.00kb
>>>> # 1.2 Routing - Loading Config
>>>> # time: 0.193375, memory: 10240.00kb
>>>> # 1.2 Routing - Searching Routes 10 times
>>>> # 1.2 Routing - Generating Routes 20 times
>>>> # time: 1.633093, memory: 3328.00kb
>>>> # 1.2 Routing - Real World Test - Accumulated data
>>>> # time: 1.826847, memory: 13568.00kb
>>>> # file: 211.00kb
>>>> [andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
>>>> symfony/test/routing_performance% php routing_performance.php
>>>> david|
>>>> grep "^#"
>>>> # testing patch david
>>>> # Creating routing yml with 1500+ routes
>>>> # david Routing - Creating Config
>>>> PHP Fatal error: Class 'splFixedArray' not found in /home/andyr/
>>>> projects/trunk-local-fs/lib/vendor/symfony/test/ >>>> routing_performance/
>>>> david/sfRoutingConfigHandlerNew.class.php on line 84
>>>> [andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
>>>> symfony/test/routing_performance% php routing_performance.php
>>>> fabian|
>>>> grep "^#"
>>>> # testing patch fabian
>>>> # Creating routing yml with 1500+ routes
>>>> # fabian Routing - Creating Config
>>>> # time: 2.712385, memory: 9472.00kb
>>>> # fabian Routing - Loading Config
>>>> # time: 0.421317, memory: 9216.00kb
>>>> # fabian Routing - Searching Routes 10 times
>>>> # fabian Routing - Generating Routes 20 times
>>>> # time: 0.814632, memory: 0.00kb
>>>> # fabian Routing - Real World Test - Accumulated data
>>>> # time: 1.236377, memory: 9216.00kb
>>>> # file: 1899.08kb
>>>> Thanks,
>>>> Andy.
>>>> On Oct 29, 5:14 pm, Fabian Lange <fabian.la...@symfony-project.com>
>>>> wrote:
>>>>> Hi David,
>>>>> feel free to send me patches for our performance test suite.
>>>>> I will incorporate them until i find a way to give access to
>>>>> everybody.
>>>>> So if you have something new i will include it.
>>>>> Ideally we have then some kind of incubation of a new way for
>>>>> routing
>>>>> to be configured. And we can benchmark them.
>>>>> So that before release 1.3 we can find a good performing solution.
>>>>> To be included in 1.3 i would like to accept only "small"
>>>>> backwards
>>>>> compatible changes that have a big net effect.
No we are not using named routes. I'm not sure why this is considered
bad practice - as far as I aware this is a feature of the framework.
The documentation IMO doesn't discourage this.
Changing to named routes would be quite an undertaking.
Thanks,
Andy
On Oct 30, 3:18 pm, Thomas Rabaix <thomas.rab...@gmail.com> wrote:
> > all the calls to url_for() collectively take 2,882,401 microseconds
> > ~93% of total request time.
> > On Oct 30, 2:40 pm, "arri...@gmail.com" <arri...@gmail.com> wrote:
> >> Hi,
> >> We use APC as the opcode cache, so all *.php files are cached by
> >> default - we had to increase the default apc.max_file_size to alow it
> >> to cache the routing cache file (config_routing.yml.php)
> >> We also use the variable cache of APC to speed up routing matches. We
> >> have this entry in factories.yml under routing:
> >> As I said, the opcode cache pretty much solves the route loading
> >> speed
> >> for us, our issue is the route matching speed without the above cache
> >> setting.
> >> On Oct 30, 12:49 pm, David Ashwood <da...@inspiredthinking.co.uk>
> >> wrote:
> >>> I ran some tests last night on the old performance framework before
> >>> downloading the latest version and starting to migrate my changes
> >>> into
> >>> the code - this is only the compression based changes - nothing
> >>> intelligent happening yet:
> >>> This is only from targeting compression - I've not looked at any
> >>> kind
> >>> of intelligent mapping/loading yet. The splArray out of habit (I've
> >>> been working on a project recently where the splDatastructures are
> >>> used). I'll ensure the next version is compatible.
> >>> How are you using/how have you configured the APC with sf?
> >>> I ask as I've been playing with some autodetection code - the
> >>> intention
> >>> being to use it if it's available - otherwise to fall back to more
> >>> conventional approaches.
> >>> On Fri, 2009-10-30 at 05:37 -0700, arri...@gmail.com wrote:
> >>>> Hi,
> >>>> Here is output from my run of the tests - I cannot run david's
> >>>> benchmark as splFixedArray is not available in my php build, though
> >>>> SPL is shown as available in php_info(). I'm running php 5.2.11.
> >>>> Re: config/cache - in 1.3 our config_routing.yml.php is 2,054,492
> >>>> bytes and consists of 628 routes. These are mostly propel routes.
> >>>> However, with APC opcode caching we find this loads very quickly
> >>>> (last
> >>>> time I benchmarked it was something like 0.06 seconds) Our
> >>>> problem is
> >>>> matching routes - if we have a page with 50 or so links, the
> >>>> majority
> >>>> of the time is spent matching the route (something like 1 second
> >>>> out
> >>>> of total request time of 1.2 seconds) If we enable the APC variable
> >>>> cache and lookup_dedicated_cache_keys once the route is matched,
> >>>> the
> >>>> performance is ok, however we have a lot of route variations and
> >>>> I can
> >>>> see this using a large amount of RAM. We cannot run without this
> >>>> cache
> >>>> as performance is just too slow. Been a while since I looked at
> >>>> this,
> >>>> but can put together some stats if it's useful?
> >>>> [andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
> >>>> symfony/test/routing_performance% php routing_performance.php 1.2|
> >>>> grep
> >>>> "^#"
> >>>> # testing patch 1.2
> >>>> # This is equivalent to the latest version from the 1.2 branch.
> >>>> # Creating routing yml with 1500+ routes
> >>>> # 1.2 Routing - Creating Config
> >>>> # time: 1.713294, memory: 2304.00kb
> >>>> # 1.2 Routing - Loading Config
> >>>> # time: 0.193375, memory: 10240.00kb
> >>>> # 1.2 Routing - Searching Routes 10 times
> >>>> # 1.2 Routing - Generating Routes 20 times
> >>>> # time: 1.633093, memory: 3328.00kb
> >>>> # 1.2 Routing - Real World Test - Accumulated data
> >>>> # time: 1.826847, memory: 13568.00kb
> >>>> # file: 211.00kb
> >>>> [andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
> >>>> symfony/test/routing_performance% php routing_performance.php
> >>>> david|
> >>>> grep "^#"
> >>>> # testing patch david
> >>>> # Creating routing yml with 1500+ routes
> >>>> # david Routing - Creating Config
> >>>> PHP Fatal error: Class 'splFixedArray' not found in /home/andyr/
> >>>> projects/trunk-local-fs/lib/vendor/symfony/test/
> >>>> routing_performance/
> >>>> david/sfRoutingConfigHandlerNew.class.php on line 84
> >>>> [andyr@cysrln03-dev-andyr-vm]~/projects/trunk-local-fs/lib/vendor/
> >>>> symfony/test/routing_performance% php routing_performance.php
> >>>> fabian|
> >>>> grep "^#"
> >>>> # testing patch fabian
> >>>> # Creating routing yml with 1500+ routes
> >>>> # fabian Routing - Creating Config
> >>>> # time: 2.712385, memory: 9472.00kb
> >>>> # fabian Routing - Loading Config
> >>>> # time: 0.421317, memory: 9216.00kb
> >>>> # fabian Routing - Searching Routes 10 times
> >>>> # fabian Routing - Generating Routes 20 times
> >>>> # time: 0.814632, memory: 0.00kb
> >>>> # fabian Routing - Real World Test - Accumulated data
> >>>> # time: 1.236377, memory: 9216.00kb
> >>>> # file: 1899.08kb
> >>>> Thanks,
> >>>> Andy.
> >>>> On Oct 29, 5:14 pm, Fabian Lange <fabian.la...@symfony-project.com>
> >>>> wrote:
> >>>>> Hi David,
> >>>>> feel free to send me patches for our performance test suite.
> >>>>> I will incorporate them until i find a way to give access to
> >>>>> everybody.
> >>>>> So if you have something new i will include it.
> >>>>> Ideally we have then some kind of incubation of a new way for
> >>>>> routing
> >>>>> to be configured. And we can benchmark them.
> >>>>> So that before release 1.3 we can find a good performing solution.
> >>>>> To be included in 1.3 i would like to accept only "small"
> >>>>> backwards
> >>>>> compatible changes that have a big net effect.
Hi David, i think the test harness is now pretty stable. Can you send me a version of your code so i can update it? Thomas, do you have any new stuff?
Regarding the Config vs. Cache issue: Lets define it as this: The cache holds parsing results, while the configuration supplies the information to generate results.
We currently target the configuration. Fabien changed that from 1.2 to 1.3 so that the runtime performance (searching, generating) is better. However the initial performance suffered from that. Our goal should eb to eliminate that, and Davids benchmarks look most promising.
What is important is that we update the routes used in the harness. So I am calling for input on the routes used in your application. Send me one or two of the most "complicated" ones.
My plan is to have a proposal ready for the BugHuntDay, where I can discuss with Fabien in person about the solution to use in sf 1.3
> Hi David,
> i think the test harness is now pretty stable. Can you send me a
> version of your code so i can update it?
> Thomas, do you have any new stuff?
> Regarding the Config vs. Cache issue:
> Lets define it as this: The cache holds parsing results, while the
> configuration supplies the information to generate results.
> We currently target the configuration. Fabien changed that from 1.2 to
> 1.3 so that the runtime performance (searching, generating) is better.
> However the initial performance suffered from that. Our goal should eb
> to eliminate that, and Davids benchmarks look most promising.
> What is important is that we update the routes used in the harness.
> So I am calling for input on the routes used in your application. Send
> me one or two of the most "complicated" ones.
> My plan is to have a proposal ready for the BugHuntDay, where I can
> discuss with Fabien in person about the solution to use in sf 1.3
I'm hoping to wander over to the PHP conf in Barcelona for a couple of
hours today but I'll have an update by Monday. I have a few versions on
the go to help compare optimisations side-by-side.
There's probably another week needed for tweaking/iterations/metrics but
there should be a week between having options and the BH day.
I'm currently looking into how best to optimize the flow of the code to
minimise repeating what's already been generated. I'm also attempting to
generate the cache in a way that will facilitate chunking of the
generated routes to allow better lazy loading of specific routes. This
should make the initial loading quicker, and handle the use case of
"module/action" routes faster.
Profiling with zend suggests that a large portion of cycles is used in
the yml processing. Not surprising - but it bears investigation as to
if the code can be optimised. Around 40% of the total test time
currently is in sfYaml*.php - half in sfYamlInline and half
sfYamlParser.
On Sat, 2009-10-31 at 11:24 +0100, Thomas Rabaix wrote: > Hello,
> I didn't have the time to look at it for now, I will try something
> before the BugHuntDay.
> Thomas
> On 30 oct. 2009, at 20:34, Fabian Lange wrote:
> > Hi David,
> > i think the test harness is now pretty stable. Can you send me a
> > version of your code so i can update it?
> > Thomas, do you have any new stuff?
> > Regarding the Config vs. Cache issue:
> > Lets define it as this: The cache holds parsing results, while the
> > configuration supplies the information to generate results.
> > We currently target the configuration. Fabien changed that from 1.2 to
> > 1.3 so that the runtime performance (searching, generating) is better.
> > However the initial performance suffered from that. Our goal should eb
> > to eliminate that, and Davids benchmarks look most promising.
> > What is important is that we update the routes used in the harness.
> > So I am calling for input on the routes used in your application. Send
> > me one or two of the most "complicated" ones.
> > My plan is to have a proposal ready for the BugHuntDay, where I can
> > discuss with Fabien in person about the solution to use in sf 1.3
Hi David,
great. I am looking forward for your proposals.
The sfYaml doesn't need to be tunes, because it is done only once in
the real app.
I included it in the test harness to make sure the routing config is
generated properly and we can use one generated yaml as base for the
test runs.
ok, if you do find some big chance to increase performance in sfYaml:
feel free to open a ticket :)
I will instrument a live app to logg all actions made with the
routing, so i get a better feeling of the usage to improve the harness
> I'm hoping to wander over to the PHP conf in Barcelona for a couple of
> hours today but I'll have an update by Monday. I have a few versions on
> the go to help compare optimisations side-by-side.
> There's probably another week needed for tweaking/iterations/metrics but
> there should be a week between having options and the BH day.
> I'm currently looking into how best to optimize the flow of the code to
> minimise repeating what's already been generated. I'm also attempting to
> generate the cache in a way that will facilitate chunking of the
> generated routes to allow better lazy loading of specific routes. This
> should make the initial loading quicker, and handle the use case of
> "module/action" routes faster.
> Profiling with zend suggests that a large portion of cycles is used in
> the yml processing. Not surprising - but it bears investigation as to
> if the code can be optimised. Around 40% of the total test time
> currently is in sfYaml*.php - half in sfYamlInline and half
> sfYamlParser.
> Regards,
> David
> On Sat, 2009-10-31 at 11:24 +0100, Thomas Rabaix wrote:
>> Hello,
>> I didn't have the time to look at it for now, I will try something
>> before the BugHuntDay.
>> Thomas
>> On 30 oct. 2009, at 20:34, Fabian Lange wrote:
>> > Hi David,
>> > i think the test harness is now pretty stable. Can you send me a
>> > version of your code so i can update it?
>> > Thomas, do you have any new stuff?
>> > Regarding the Config vs. Cache issue:
>> > Lets define it as this: The cache holds parsing results, while the
>> > configuration supplies the information to generate results.
>> > We currently target the configuration. Fabien changed that from 1.2 to
>> > 1.3 so that the runtime performance (searching, generating) is better.
>> > However the initial performance suffered from that. Our goal should eb
>> > to eliminate that, and Davids benchmarks look most promising.
>> > What is important is that we update the routes used in the harness.
>> > So I am calling for input on the routes used in your application. Send
>> > me one or two of the most "complicated" ones.
>> > My plan is to have a proposal ready for the BugHuntDay, where I can
>> > discuss with Fabien in person about the solution to use in sf 1.3
I can attest for the bad memory in generating routes. I am using a lot
of url_for() in a batch script to generate the sitemap of the site
(100,000) URLs and the script dies out of memory (256MB)
Fatal error: Allowed memory size of 268435456 bytes exhausted (tried
to allocate 4864 bytes) in symfony/routing/sfRoute.class.php(229) :
runtime-created function on line 1
I will be able to test the changes you are making if they will
actually improve the memory usage in the generating stage.
On 31 Окт, 21:19, Fabian Lange <fabian.la...@symfony-project.com>
wrote:
> Hi David,
> great. I am looking forward for your proposals.
> The sfYaml doesn't need to be tunes, because it is done only once in
> the real app.
> I included it in the test harness to make sure the routing config is
> generated properly and we can use one generated yaml as base for the
> test runs.
> ok, if you do find some big chance to increase performance in sfYaml:
> feel free to open a ticket :)
> I will instrument a live app to logg all actions made with the
> routing, so i get a better feeling of the usage to improve the harness
> > I'm hoping to wander over to the PHP conf in Barcelona for a couple of
> > hours today but I'll have an update by Monday. I have a few versions on
> > the go to help compare optimisations side-by-side.
> > There's probably another week needed for tweaking/iterations/metrics but
> > there should be a week between having options and the BH day.
> > I'm currently looking into how best to optimize the flow of the code to
> > minimise repeating what's already been generated. I'm also attempting to
> > generate the cache in a way that will facilitate chunking of the
> > generated routes to allow better lazy loading of specific routes. This
> > should make the initial loading quicker, and handle the use case of
> > "module/action" routes faster.
> > Profiling with zend suggests that a large portion of cycles is used in
> > the yml processing. Not surprising - but it bears investigation as to
> > if the code can be optimised. Around 40% of the total test time
> > currently is in sfYaml*.php - half in sfYamlInline and half
> > sfYamlParser.
> > Regards,
> > David
> > On Sat, 2009-10-31 at 11:24 +0100, Thomas Rabaix wrote:
> >> Hello,
> >> I didn't have the time to look at it for now, I will try something
> >> before the BugHuntDay.
> >> Thomas
> >> On 30 oct. 2009, at 20:34, Fabian Lange wrote:
> >> > Hi David,
> >> > i think the test harness is now pretty stable. Can you send me a
> >> > version of your code so i can update it?
> >> > Thomas, do you have any new stuff?
> >> > Regarding the Config vs. Cache issue:
> >> > Lets define it as this: The cache holds parsing results, while the
> >> > configuration supplies the information to generate results.
> >> > We currently target the configuration. Fabien changed that from 1.2 to
> >> > 1.3 so that the runtime performance (searching, generating) is better.
> >> > However the initial performance suffered from that. Our goal should eb
> >> > to eliminate that, and Davids benchmarks look most promising.
> >> > What is important is that we update the routes used in the harness.
> >> > So I am calling for input on the routes used in your application. Send
> >> > me one or two of the most "complicated" ones.
> >> > My plan is to have a proposal ready for the BugHuntDay, where I can
> >> > discuss with Fabien in person about the solution to use in sf 1.3
> I can attest for the bad memory in generating routes. I am using a lot
> of url_for() in a batch script to generate the sitemap of the site
> (100,000) URLs and the script dies out of memory (256MB)
> Fatal error: Allowed memory size of 268435456 bytes exhausted (tried
> to allocate 4864 bytes) in symfony/routing/sfRoute.class.php(229) :
> runtime-created function on line 1
> I will be able to test the changes you are making if they will
> actually improve the memory usage in the generating stage.
> On 31 Окт, 21:19, Fabian Lange <fabian.la...@symfony-project.com>
> wrote:
>> Hi David,
>> great. I am looking forward for your proposals.
>> The sfYaml doesn't need to be tunes, because it is done only once in
>> the real app.
>> I included it in the test harness to make sure the routing config is
>> generated properly and we can use one generated yaml as base for the
>> test runs.
>> ok, if you do find some big chance to increase performance in sfYaml:
>> feel free to open a ticket :)
>> I will instrument a live app to logg all actions made with the
>> routing, so i get a better feeling of the usage to improve the harness
>> > I'm hoping to wander over to the PHP conf in Barcelona for a couple of
>> > hours today but I'll have an update by Monday. I have a few versions on
>> > the go to help compare optimisations side-by-side.
>> > There's probably another week needed for tweaking/iterations/metrics but
>> > there should be a week between having options and the BH day.
>> > I'm currently looking into how best to optimize the flow of the code to
>> > minimise repeating what's already been generated. I'm also attempting to
>> > generate the cache in a way that will facilitate chunking of the
>> > generated routes to allow better lazy loading of specific routes. This
>> > should make the initial loading quicker, and handle the use case of
>> > "module/action" routes faster.
>> > Profiling with zend suggests that a large portion of cycles is used in
>> > the yml processing. Not surprising - but it bears investigation as to
>> > if the code can be optimised. Around 40% of the total test time
>> > currently is in sfYaml*.php - half in sfYamlInline and half
>> > sfYamlParser.
>> > Regards,
>> > David
>> > On Sat, 2009-10-31 at 11:24 +0100, Thomas Rabaix wrote:
>> >> Hello,
>> >> I didn't have the time to look at it for now, I will try something
>> >> before the BugHuntDay.
>> >> Thomas
>> >> On 30 oct. 2009, at 20:34, Fabian Lange wrote:
>> >> > Hi David,
>> >> > i think the test harness is now pretty stable. Can you send me a
>> >> > version of your code so i can update it?
>> >> > Thomas, do you have any new stuff?
>> >> > Regarding the Config vs. Cache issue:
>> >> > Lets define it as this: The cache holds parsing results, while the
>> >> > configuration supplies the information to generate results.
>> >> > We currently target the configuration. Fabien changed that from 1.2 to
>> >> > 1.3 so that the runtime performance (searching, generating) is better.
>> >> > However the initial performance suffered from that. Our goal should eb
>> >> > to eliminate that, and Davids benchmarks look most promising.
>> >> > What is important is that we update the routes used in the harness.
>> >> > So I am calling for input on the routes used in your application. Send
>> >> > me one or two of the most "complicated" ones.
>> >> > My plan is to have a proposal ready for the BugHuntDay, where I can
>> >> > discuss with Fabien in person about the solution to use in sf 1.3
|
